Set max_tokens on model-level #2375
hofq
started this conversation in
Help Wanted
Replies: 2 comments 5 replies
-
This is planned and have been working on this |
Beta Was this translation helpful? Give feedback.
0 replies
-
Also if you name your model according to its version, it won’t be an issue. You can decouple the deployment name from the model name for more control by using the librechat.yaml file as well for azure configuration https://docs.librechat.ai/install/configuration/azure_openai.html |
Beta Was this translation helpful? Give feedback.
5 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hey there,
ive wanted to ask a quick question:
how can i set max_tokens for a specific model?
We use the Azure OpenAI Endpoint with gpt-35-turbo in Version 1106. The Model's Token Limit from OpenAI Docs is 16385, but Librechat is Returning an Error on anything higher than 4092.
How can i overwrite that?
Beta Was this translation helpful? Give feedback.
All reactions