Testhost is blocked on system error message dialog

[riQQ]

Description

The testhost is blocked on a system error message dialog. I haven't analyzed the cause of the system error yet, but the problem here is that a system error dialog is shown at all. Running the tests in a build pipeline via Azure DevOps / VSTest task / vstest.console the task just hangs until the timeout is hit. This issue can only be diagnosed with access to the CI system to see that it's the system error dialog that causes the timeout.

Steps to reproduce

So far I haven't managed to produce a small sample that I can share. I'll update this post if I manage to produce a self contained repro.

Expected behavior

Testhost crashes and the error is logged.

Actual behavior

Testhost is blocked on system error message dialog

Diagnostic logs

There is no mention of the error in the logs, just polling on both sides.

vstest.console console output after clicking OK on the system error dialog

The active test run was aborted. Reason: Test host process crashed

vstest.console diagnostic log

TpTrace Verbose: 0 : 26720, 10, 2024/07/18, 14:01:59.835, 180619017190, vstest.console, TcpClientExtensions.MessageLoopAsync: Polling on remoteEndPoint: 127.0.0.1:61110 localEndPoint: 127.0.0.1:61109 after 1006 ms

testhost

TpTrace Verbose: 0 : 13916, 11, 2024/07/18, 14:04:31.099, 182131653244, testhost.net472.exe, TcpClientExtensions.MessageLoopAsync: Polling on remoteEndPoint: 127.0.0.1:61109 localEndPoint: 127.0.0.1:61110 after 1013 ms

Environment

Windows 10
VSTest from Visual Studio installation
testhost.net472.exe / testhost.net472.x86.exe version 17.10.0-release-24177-07
vstest.console.exe 17.10.0-release-24177-07 ()

[nohwnd]

I don't think there is a way for us to influence this. It is the same deal with debug assertion messages (Debug.Assert). Unless the underlying framework implements a trace listener that does not forward the assert failure, the default will show an assertion pop-up, and will block the run.

Is it happening in any run? My impression is that when the session is not interactive you don't get these pop-ups, and imho interactive session is set only when you say you have UI tests in the mix (at least when using VSTest AzDO task).

[nohwnd]

We most likely cannot affect this, and it would be a new feature, or more likely a responsibility of the testing framework (e.g. mstest, nunit, xunit) rather than responsibility of this platform.

I would be curious if you come up with a solution, if yes we could consider it for testing platform / mstest (both residing in https://github.com/microsoft/testfx). 😊

But at the moment this repo (vstest) is not accepting any new features.

[riQQ]

My impression is that when the session is not interactive you don't get these pop-ups, and imho interactive session is set only when you say you have UI tests in the mix (at least when using VSTest AzDO task).

We're using Azure DevOps Server with our own runners with interactive sessions even if the test set doesn't contain any UI tests.

I would be curious if you come up with a solution, if yes we could consider it for testing platform / mstest (both residing in https://github.com/microsoft/testfx). 😊

I thought about calling SetErrorMode(GetErrorMode () | SEM_NOGPFAULTERRORBOX)); or something along these lines in the testhost. What do you think about that? Should I open an issue over there?