Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update README.md #620

Merged
merged 1 commit into from
Dec 18, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,7 @@ Dynamic settings can include the following parameters:
| keys.<core_key> | `project`: Project name is assigned to this key. **Required** <br />`role`: a role to be assigned to the key. **Note:** a key is invalid if `role` and `roles` are missed. <br />`roles`: a list of roles to be assigned to the key. **Note:** a key is invalid if `role` and `roles` are missed. <br/> `secured`: the flag indicates if the key is secured. If it's set to `true` user request and deployment response won't be saved to the prompt log storage. |
| roles | API key or user roles. Each role may have limits to be associated with applications, models, assistants or addons. Refer to [API Keys](https://github.com/epam/ai-dial/blob/main/docs/Roles%20and%20Access%20Control/3.API%20Keys.md) to learn more. |
| roles.<role_name> | `limits`: Limits for models, applications, or assistants. **Note:** it is necessary to define this for a role. |
| roles.<role_name>.limits | `minute`: Total tokens per minute limit sent to the model, managed via floating window approach for well-distributed rate limiting. If it's not set the default value is unlimited<br />`day`: Total tokens per day limit sent to the model, managed via floating window approach for balanced rate limiting. **Note**: you can skip these parameters to apply their default value - unlimited. |
| roles.<role_name>.limits | `minute`: Total tokens per minute limit sent to the model, managed via floating window approach for well-distributed rate limiting. If it's not set the default value is unlimited<br />`day`: Total tokens per day limit sent to the model, managed via floating window approach for balanced rate limiting.<br />`week`: Total tokens per week limit sent to the model, managed via floating window approach for balanced rate limiting.<br />`month`: Total tokens per month limit sent to the model, managed via floating window approach for balanced rate limiting.<br />**Note**: you can skip these parameters to apply their default value - unlimited. |
| retriableErrorCodes | List of retriable error codes for handling outages at LLM providers. |

## License
Expand Down
Loading