Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix IllegalStateException in HttpChannelState.completed() after abort() #12545

Merged
merged 2 commits into from
Nov 26, 2024

Conversation

lorban
Copy link
Contributor

@lorban lorban commented Nov 18, 2024

Fixes #12520

The following stack trace is logged when a certain sequence of events is happening in the FSM:

java.lang.IllegalStateException: s=HANDLING rs=COMPLETED os=ABORTED is=IDLE awp=false se=false i=false al=0
	at Jenkins Main ClassLoader//org.eclipse.jetty.ee9.nested.HttpChannelState.completed(HttpChannelState.java:1025)
	at Jenkins Main ClassLoader//org.eclipse.jetty.ee9.nested.HttpChannel.lambda$handle$0(HttpChannel.java:634)
	at Jenkins Main ClassLoader//org.eclipse.jetty.util.Callback$3.succeeded(Callback.java:170)
	at Jenkins Main ClassLoader//org.eclipse.jetty.ee9.nested.HttpOutput.complete(HttpOutput.java:493)
	at Jenkins Main ClassLoader//org.eclipse.jetty.ee9.nested.Response.completeOutput(Response.java:990)
	at Jenkins Main ClassLoader//org.eclipse.jetty.ee9.nested.HttpChannel.handle(HttpChannel.java:634)
	at Jenkins Main ClassLoader//org.eclipse.jetty.ee9.nested.ContextHandler$CoreContextHandler$CoreToNestedHandler.handle(ContextHandler.java:2873)
	at Jenkins Main ClassLoader//org.eclipse.jetty.server.handler.ContextHandler.handle(ContextHandler.java:1060)
	at Jenkins Main ClassLoader//org.eclipse.jetty.server.Server.handle(Server.java:182)
	at Jenkins Main ClassLoader//org.eclipse.jetty.server.internal.HttpChannelState$HandlerInvoker.run(HttpChannelState.java:662)
	at Jenkins Main ClassLoader//org.eclipse.jetty.util.thread.Invocable$ReadyTask.run(Invocable.java:168)
	at Jenkins Main ClassLoader//org.eclipse.jetty.http2.server.internal.HttpStreamOverHTTP2$1.run(HttpStreamOverHTTP2.java:135)
	at Jenkins Main ClassLoader//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:478)
	at Jenkins Main ClassLoader//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:441)
	at Jenkins Main ClassLoader//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:293)
	at Jenkins Main ClassLoader//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:201)
	at Jenkins Main ClassLoader//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:311)
	at Jenkins Main ClassLoader//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:979)
	at Jenkins Main ClassLoader//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1209)
	at Jenkins Main ClassLoader//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1164)
	at java.base/java.lang.Thread.run(Thread.java:1583)

These are the calls and states that triggers this bug:

1. abort() / s=HANDLING rs=COMPLETING os=ABORTED is=IDLE awp=false se=false i=false al=0 - failure is java.io.IOException: Insufficient content written
2. completed() / s=HANDLING rs=COMPLETED os=ABORTED is=IDLE awp=false se=false i=false al=0 - throws IllegalStateException: s=HANDLING rs=COMPLETED os=ABORTED is=IDLE awp=false se=false i=false al=0

abort() incorrectly moves the request state to COMPLETED when it changes the state to WOKEN: this transition requires a redispatch that eventually calls completed() while the request state already is at its terminal position.

See #11210 as the same problem was fixed in EE10, but the fact that EE9 might have a similiar problem was blurred by the fact that the original issue had to do with GZIP which was buggy too.

@lorban lorban marked this pull request as ready for review November 18, 2024 14:59
@lorban lorban self-assigned this Nov 18, 2024
@lorban lorban added the Bug For general bugs on Jetty side label Nov 18, 2024
@lorban lorban requested a review from sbordet November 18, 2024 14:59
…11210)

Signed-off-by: Ludovic Orban <lorban@bitronix.be>
@lorban lorban force-pushed the fix/jetty-12.0.x/12520-fix-abort-complete-fsm branch from ef453d6 to f61aee2 Compare November 20, 2024 13:18
@gregw gregw self-requested a review November 20, 2024 19:55
Copy link
Contributor

@gregw gregw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to have a test for this use-case.

@lorban lorban merged commit 7f61162 into jetty-12.0.x Nov 26, 2024
10 checks passed
@lorban lorban deleted the fix/jetty-12.0.x/12520-fix-abort-complete-fsm branch November 26, 2024 11:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For general bugs on Jetty side
Projects
Status: ✅ Done
Development

Successfully merging this pull request may close these issues.

Numerous stack traces logged at warning level when running under HTTP/2 (regression in 12.0.15)
3 participants