TestCategory no longer works for VsTest

[aaronportier]

I changed my appium dotnet version and had to add yaml run settings files for the cloud provider I am using. I am running this in Azure Pipeline with VsTest. The output seems like it knows how many tests should run with the Category but gives this message:
now IsValidServiceResponse: Received None command..Service Workflow is not active
Creating run for selected test assemblies with following parameters
SourceFilter: ThisWontBeUsed TestCaseFilter: TestCategory=UserSubscription
Run title: TestRun_TestAutomationBSIntegration_Release-598

Why will it not be used? I even tried adding to the Runsettigns filter (TestCategory = UserSubscription) but that still gives me the same error

[nohwnd]

I don't think I follow what is not working. Can you please give a list of simple steps to reproduce this?

[aaronportier]

For a repo of steps just use the above chnages to your appium project and then TestCategory added in Test Criteria for vsTest will no longer work.
I can see in the logs it says "This Wont Be Used" but why will it not be used?

Creating run for selected test assemblies with following parameters
SourceFilter: ThisWontBeUsed TestCaseFilter: TestCategory=UserSubscription
Run title: TestRun_TestAutomationBSIntegration_Release-606

[aaronportier]

I am using Run Settings and I have tried putting User Subscription in there as well but still it runs all tests not just the category I choose. The correct amount of tests is 16 but they all run without Category filtering them.
2023-12-14T19:59:47.0011882Z Test filter criteria : TestCategory=UserSubscription...
2023-12-14T19:59:47.9808239Z Number of test cases per batch : 16...
Creating run for selected test assemblies with following parameters
SourceFilter: ThisWontBeUsed TestCaseFilter: TestCategory=UserSubscription
Run title: TestRun
In the VsTest-testAssemblies on Azure Pipelines my Agent Job I have Test filter criteria TestCategory=UserSubscription.
What am I missing why will it not filter by Category for me? It used to work until I had to create Run Settings file and upgrade my dotnet appium to the 5.0 version.

[aaronportier]

Does anyone know where this line SourceFilter: ThisWontBeUsed might be coming from? In the past I never saw it now I have run setting files but they should not be doing anything to source filters.

[aaronportier]

I was able to get everything to run by turning Batch off. Unfortunately turning off Batch options makes my pipeline unable to re-run failed tests.I know that was a work around was the issue ever fixed.

[aaronportier]

Is anyone else experiencing or reporting this issue. It was working but now its not and I cant figure out why? The batching if clicked will not run using your Category Filter but If you turn batching off it will not re-run failed tests. Saying that it could not find them.

[aaronportier]

This is the state of my VSTest any ideas on next steps?

[nohwnd]

Could you sum this up into a simple repro please?

I think what is happening is that running with batches is running your tests with DTAgent, and that ignores commandline parameters like the filter.

Are you setting the filter on commandline or in runsettings file?

[aaronportier]

Command line. It is being run in Vstest with the filter in the test.

…

On Mon, Jul 8, 2024, 6:25 AM Jakub Jareš ***@***.***> wrote: Could you sum this up into a simple repro please? I think what is happening is that running with batches is running your tests with DTAgent, and that ignores commandline parameters like the filter. Are you setting the filter on commandline or in runsettings file? — Reply to this email directly, view it on GitHub <#4782 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGVOYNT4F7TDPBXZYJRAYLZLJSKTAVCNFSM6AAAAABALSMPAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJTGYZDSMZTGA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[nohwnd]

Please share a repro project and a command that shows the problem.