-
Notifications
You must be signed in to change notification settings - Fork 350
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle faulty TCP connection between XHarness and Apple devices #11700
Comments
/cc @mandel-macaque |
[Async-Triage] Ilya suggested we take this through the FR and I was going to do it during my last rotation but the FR was super busy that week and I didn't get around to it. Ilya also mentioned it might be useful for someone else to have a go at this so that we spread the knowledge about mobile devices a bit while I would support the person. I tried to spec out the issue in enough detail for that to be hopefully possible. |
Premek, can you discuss the impact of this, ie the business impact (more what benefit would introducing the new error code vs just retrying on APP_CRASH)? |
The impact is a bit described in the linked issue where a customer reported failing rolling builds where this just happens often enough that it matters. This issue actually has been around since August and we thought we can ignore it but it just happens frequently enough that we shouldn't ignore it. As Jan describes in the issue - ignoring all Unfortunately, we cannot turn the original issue into a As an additional example of a potential business impact, we've just caught a big one similar to this 2 weeks ago which almost went into the |
Side note: I am on FR the next week |
@garath, what is the current status of this work item? |
Wasn't it @jonfortescue working on this? |
John handed off to Stu last week |
Kudos to you @AlitzelMendez for not being afraid to jump into the mobile space. Good work! |
Thanks @AlitzelMendez, assigning to myself and will monitor this through out the week |
Actually, will keep the customer issue up for tracking of this: #11683 |
Context
When we run mobile device tests on Apple platforms, we create a TCP tunnel through the USB cable that the AppleTV / iPhone is attached by. The TCP tunnel is created by a tool called mlaunch that XHarness uses to talk to the device. A separate mlaunch process is launched on the side that creates the tunnel and XHarness reads its stdout.
Problem with TCP
It can happen that the TCP tunnel between XHarness and the device fails:
It can also happen that the tunnel is created but the device cannot connect to it (desribed in dotnet/xharness#934).
When either of thees happens, the app running on the device fails to connect as a plan B just writes into stdout instead the TCP connection (the stdout is then in the
net.dot.System.Runtime.Tests.log
log). So while the app did performs the actual unit tests fine, the run is considered anAPP_CRASH
because the app never talked back.Goal
The subject of this issue is to not solve the TCP flakiness as that is somewhere between iOS, MacOS, mlaunch and the .NET runtime. The problem we want to solve here is that we qualify the TCP as
APP_CRASH
which puts it in the same boat as when the app actually crashes.Instead, we'd like to recognize the TCP tunnel being the problem and exit with some new exit code (e.g.
TCP_CONNECTION_FAILED
).We can then set retries for this specific case as this can be considered an infra failure and let
APP_CRASH
not retry.To summarize:
TCP_CONNECTION_FAILED
and return it instead ofAPP_CRASH
when TCP diesAPP_CRASH
(see #11689)Development instructions
It should be easy to stage a repro. You can do it like this:
To repro, you can just send the wrong port to the app (e.g.
deviceListenerPort + 1
) and this way the phone/simulator won't connect.You can use the XHarness E2E tests to create a Helix job from
dotnet/xharness
: https://github.com/dotnet/xharness/blob/main/tools/run-e2e-test.ps1You should be able to work with this. More details are also here: dotnet/xharness#934
Release Note Category
Release Note Description
Improved handling of faulty TCP connection between XHarness and Apple devices
The text was updated successfully, but these errors were encountered: