Related to #425, #263
Is there a consensus on implementing token usage tracking?
The most bare-bones option would probably be to just add a get_usage() method to BaseLanguageModel which returns the accumulated token usage of that model. Then that would be implemented for the supported models and a user could then call that method with a model from factory.create_model.
But maybe it'd be better to have a more general get_metadata() method for BaseLanguageModel that happens to return usage as part of it for the supported models.
For the general metadata option, maybe the output looks something like
{
usage: {
intput_tokens: int,
output_tokens: int
}
}
I'd be happy to contribute to this feature myself.
Related to #425, #263
Is there a consensus on implementing token usage tracking?
The most bare-bones option would probably be to just add a
get_usage()method to BaseLanguageModel which returns the accumulated token usage of that model. Then that would be implemented for the supported models and a user could then call that method with a model from factory.create_model.But maybe it'd be better to have a more general
get_metadata()method forBaseLanguageModelthat happens to return usage as part of it for the supported models.For the general metadata option, maybe the output looks something like
I'd be happy to contribute to this feature myself.