graceful_shutdown doesn't shut down gracefully #114
Comments
|
I should probably add that this problem surfaced with the upgrade from |
droundy
changed the title
phlip9
added a commit
to lexe-app/axum-server
that referenced
this issue
May 18, 2024
The hyper v1 migration PR (programatik29#93) accidentally (?) removed the call to `serve_future.graceful_shutdown()`, which prevents the server from shutting down until all client _connections_ close on their own (or we hit the hard shutdown timeout). In practice, this meant we always hit the hard shutdown timeout, since most clients use connection pools with connection keepalive. With this change, graceful shutdown will now correctly stop handling any new requests after it's triggered, while letting existing requests complete (within the timeout of course). Fixes programatik29#114
phlip9
added a commit
to lexe-app/axum-server
that referenced
this issue
May 20, 2024
The hyper v1 migration PR (programatik29#93) accidentally (?) removed the call to `serve_future.graceful_shutdown()`, which prevents the server from shutting down until all client _connections_ close on their own (or we hit the hard shutdown timeout). In practice, this meant we always hit the hard shutdown timeout, since most clients use connection pools with connection keepalive. With this change, graceful shutdown will now correctly stop handling any new requests after it's triggered, while letting existing requests complete (within the timeout of course). Fixes programatik29#114
phlip9
added a commit
to lexe-app/axum-server
that referenced
this issue
May 20, 2024
The hyper v1 migration PR (programatik29#93) accidentally (?) removed the call to `serve_future.graceful_shutdown()`, which prevents the server from shutting down until all client _connections_ close on their own (or we hit the hard shutdown timeout). In practice, this meant we always hit the hard shutdown timeout, since most clients use connection pools with connection keepalive. With this change, graceful shutdown will now correctly stop handling any new requests after it's triggered, while letting existing requests complete (within the timeout of course). Fixes programatik29#114
phlip9
added a commit
to lexe-app/axum-server
that referenced
this issue
May 20, 2024
The hyper v1 migration PR (programatik29#93) accidentally (?) removed the call to `serve_future.graceful_shutdown()`, which prevents the server from shutting down until all client _connections_ close on their own (or we hit the hard shutdown timeout). In practice, this meant we always hit the hard shutdown timeout, since most clients use connection pools with connection keepalive. With this change, graceful shutdown will now correctly stop handling any new requests after it's triggered, while letting existing requests complete (within the timeout of course). Fixes programatik29#114
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'm seeing that there is always a single connection registered if the
Watcherfor a connection hasn't yet been dropped whengraceful_shutdownis called (which is the point ofgraceful_shutdown!).connection made, and so
graceful_shutdowndoesn't exit until the timeout happens.I've reproduced this both with
axum-server0.6.0, and with the currentmasterbranch. I added prints to confirm which code branches are getting triggered. This is readily triggered by integration tests in my code (which I cannot share), which will uniformly fail if I don't insert a brief wait between when I receive a response and when I send a SIGINT to the process.Specifically, I'm seeing that after the code reaches this line:
https://github.com/programatik29/axum-server/blob/master/src/server.rs#L207
the
serve_futurein the innerselect!doesn't ever finish.If I wait a millisecond before sending the SIGINT, then the connection is marked as completed and graceful shutdown exits cleanly. Except of course that this is the scenario where one wants to gracefully shut down.
The text was updated successfully, but these errors were encountered: