Skip to content

Commit

Permalink
collab: Temporarily bypass LLM rate limiting for staff (#16089)
Browse files Browse the repository at this point in the history
This PR makes it so staff members will be exempt from rate limiting by
the LLM service.

This is just a temporary measure until we can tweak the rate-limiting
heuristics.

Staff members are still subject to upstream LLM provider rate limits.

Release Notes:

- N/A
  • Loading branch information
maxdeviant authored Aug 11, 2024
1 parent 492f6b9 commit 3140d6c
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions crates/collab/src/llm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -400,6 +400,11 @@ async fn check_usage_limit(
];

for (usage, limit, resource) in checks {
// Temporarily bypass rate-limiting for staff members.
if claims.is_staff {
continue;
}

if usage > limit {
return Err(Error::http(
StatusCode::TOO_MANY_REQUESTS,
Expand Down

0 comments on commit 3140d6c

Please sign in to comment.