Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better doc for different between timeout and client_timeout of grpc_client.infer #7369

Open
ShuaiShao93 opened this issue Jun 24, 2024 · 0 comments

Comments

@ShuaiShao93
Copy link

Is your feature request related to a problem? Please describe.
In the doc, it says for timeout, it's only only respected by the model that is configured with dynamic batching, and the server can take model-specific actions. But it's not clear what model-specific actions are. Does it terminate the request for all models? How is this different from client_timeout?

Describe the solution you'd like
Update the doc with the details below:

  • What are model-specific actions?
  • Does client_timeout also terminate all requests?
  • Why is timeout only respected by the model that is configured with dynamic batching?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant