No possibility to run tests with long VSTestTestCaseFilter

[ch1seL]

Description

I think this problem connected with Win API. CreateProccessA function has a limitation on lpCommandLine parameter. lpCommandLine. The maximum length of this string is 32,767 characters

Steps to reproduce

• compose file with hundreds of test case filters, for example, tests.rsp:
  /p:VSTestTestCaseFilter=(FullyQualifiedName="Test1")|(FullyQualifiedName="Test2")|...|(FullyQualifiedName="Test9999")
• run dotnet test @tests.rsp

Expected behavior

tests should be run

Actual behavior

task failed unexpectedly

Diagnostic logs

C:\Program Files\dotnet\sdk\5.0.103\Microsoft.TestPlatform.targets(32,5): error MSB4018: System.ComponentModel.Win32Exception (206): The filename or extension is too long.
C:\Program Files\dotnet\sdk\5.0.103\Microsoft.TestPlatform.targets(32,5): error MSB4018:    at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo)
C:\Program Files\dotnet\sdk\5.0.103\Microsoft.TestPlatform.targets(32,5): error MSB4018:    at System.Diagnostics.Process.Start()
C:\Program Files\dotnet\sdk\5.0.103\Microsoft.TestPlatform.targets(32,5): error MSB4018:    at Microsoft.TestPlatform.Build.Tasks.VSTestForwardingApp.Execute()
C:\Program Files\dotnet\sdk\5.0.103\Microsoft.TestPlatform.targets(32,5): error MSB4018:    at Microsoft.TestPlatform.Build.Tasks.VSTestTask.Execute()
C:\Program Files\dotnet\sdk\5.0.103\Microsoft.TestPlatform.targets(32,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
C:\Program Files\dotnet\sdk\5.0.103\Microsoft.TestPlatform.targets(32,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask)

Environment

OS Name: Microsoft Windows Server 2016 Standard
OS Version: 10.0.14393 N/A Build 14393
.Net SDK: 5.0.103
MSBUILD: 16.8.3+39993bd9d for .NET

[Sanan07]

Limit by OS, not by TestPlatform

[ch1seL]

Limit by OS, not by TestPlatform

Absolutely, but TestPlatform can import a huge filter from a file as an option to avoid restrictions.
For example, NUnit console runner support --testlist option to pass a list of tests as a file.
https://docs.nunit.org/articles/nunit/running-tests/Console-Command-Line.html#options

[rsaugier]

Some people have very large test suites!
Being able to specify easily a large sub-set of tests to execute seems to be a valid use case to me (think of splitting for parallel execution).
This is not possible with this command-line restriction, neither with /Tests (which by the way specifies search patterns, not exact test method names) nor /TestCaseFilter ...
There might be a workarounds here but I think it's undocumented (added just for VS ?), so probably fragile...

A /TestCaseFilterFile:plaintext_file_with_list_of_filters.txt would be nice.

[nohwnd]

The solution linked above is stable, it is not just for VS: #2273 (comment)

Another option is using response files. They currently don't work with dotnet test and dotnet vstest, but work just fine for vstest.console. Tracking that bug here: #3513

[rsaugier]

Ok, great, Thanks @nohwnd for the clarification!