You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
In the doc, it says for timeout, it's only only respected by the model that is configured with dynamic batching, and the server can take model-specific actions. But it's not clear what model-specific actions are. Does it terminate the request for all models? How is this different from client_timeout?
Describe the solution you'd like
Update the doc with the details below:
What are model-specific actions?
Does client_timeout also terminate all requests?
Why is timeout only respected by the model that is configured with dynamic batching?
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
In the doc, it says for timeout, it's only only respected by the model that is configured with dynamic batching, and the server can take model-specific actions. But it's not clear what model-specific actions are. Does it terminate the request for all models? How is this different from client_timeout?
Describe the solution you'd like
Update the doc with the details below:
The text was updated successfully, but these errors were encountered: