Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TogetherSolver #1502

Merged
merged 1 commit into from
Mar 22, 2024
Merged

TogetherSolver #1502

merged 1 commit into from
Mar 22, 2024

Conversation

thesofakillers
Copy link
Contributor

@thesofakillers thesofakillers commented Mar 21, 2024

This PR contributes a TogetherSolver class, a solver for using models served by the Together AI API

Because Together supports the OpenAI python sdk, we simply create a subclass of the OpenAISolver, overriding some functionality. There is therefore some refactoring of the OpenAISolver included in this PR to facilitate this code sharing.

At the moment, we support the models specified in evals/registry/solvers/together.yaml, but in principle most models offered from the Together AI API can easily be added

Notes:

  • logit biasing not supported by the Together API due to a lack of a unified tokenizer a la tiktoken from openai
  • For the same reason, checking for context length limits not supported

Co-authored-by: Chan Jun Shern <chanjunshern@gmail.com>
Co-authored-by: Ian McKenzie <ian.mckenzie@c-openai.com>
Copy link
Collaborator

@JunShern JunShern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this! This allows us to test lots more models including Llama, Mixtral, etc.

@JunShern JunShern merged commit 5805c20 into openai:main Mar 22, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants