Feature Request: ability to filter out tests from default test run

[alefranz]

Background

One of the great aspects .NET tooling is that you can clone a repository and run dotnet test and get your solution build and the tests to run. Nice and quick!
However there are situation when tests can not run that easily, a common one being integration tests that require your service/application to run.

Description

It would be nice to have the ability to mark such tests so that they can be excluded by default from a test run, either it being dotnet test or your "Run All Tests" in Visual Studio (or other IDEs).
Goal is to improve the Dev Ex when jumping on a unfamiliar code base.

Current Behavior

Currently there is no way to achieve this, except adding a bunch of customization in your repository, e.g. scripts for when running from command line, solution filters (SLNF) for Visual Studio and so on.
This is loosely related to #2818, as having the ability to just mark the tests that you don't want to run is definitely a step in the right direction, but it would be nice to be able to define this as default behavior.

Proposal

Some possible solutions:

• Define at solution level a default filter, which will be used by dotnet test and Visual Studio when none is specified.
• Introduce a concept at platform level of tests excluded from default runs, and leverage an attribute to exclude tests, similar to ExcludeFromCodeCoverageAttribute so you can easily exclude a class, test or entire assembly
  e.g. to exclude integration tests project you could add

    <ItemGroup>
        <AssemblyAttribute Include="Microsoft.NET.Test.Sdk.ExcludeFromDefaultTestRunAttribute" />
    </ItemGroup>

[jakubpas]

Vote up

[Evangelink]

Hey @alefranz and @jakubpas, thanks for the suggestion.

While this feature is being discussed, you can use some of the already existing filters and document it in your repo readme. For example you can use TestCategory attribute or filter by namespace (or assembly). See docs about existing filters https://learn.microsoft.com/dotnet/core/testing/selective-unit-tests?pivots=mstest and https://learn.microsoft.com/dotnet/core/tools/dotnet-test#filter-option-details

[nohwnd]

The best you can approximate this is by specifying runsettings in profile, and have the runsettings have a filter.

Otherwise this is a new feature and won't be implemented, we are focusing on adding new features to Testing.Platform instead. https://aka.ms/testingplatform