Skip to content

Feature Request: Token Usage Tracking (or model metadata?) Support #475

Description

@g-jensen

Related to #425, #263

Is there a consensus on implementing token usage tracking?

The most bare-bones option would probably be to just add a get_usage() method to BaseLanguageModel which returns the accumulated token usage of that model. Then that would be implemented for the supported models and a user could then call that method with a model from factory.create_model.

But maybe it'd be better to have a more general get_metadata() method for BaseLanguageModel that happens to return usage as part of it for the supported models.

For the general metadata option, maybe the output looks something like

{
  usage: {
    intput_tokens: int,
    output_tokens: int
  }
}

I'd be happy to contribute to this feature myself.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions