Introducing MSTest SDK – Improved Configuration & Flexibility

Marco Rossignoli

Amaury Levé

April 11th, 202419 5

We are excited to announce the new MSTest SDK built on top of the MSBuild Project SDK system. It is designed to give you a better experience for testing with MSTest by making project configuration easier through sensible defaults and flexible options.

This new experience was built on top of the recently introduced MSTest runner (check out the announcement post) to further simplify your experience. This new runner, a lightweight, reliable, and performant way to run MSTest tests, is shipped as dependency of MSTest.TestAdapter NuGet package. The runner and its extensions consist of multiple NuGet packages to provide an extensible, flexible, and configurable test running experience. However, being customizable can result in many questions: What are the recommended extensions? What are the correct defaults? How do I align the versions? This is where the MSTest SDK comes in.

Getting started with MSTest SDK

Getting started using the new MSTest SDK is easy. Just create a MSTest project (or update an existing MSTest project) and replace the content of the .csproj file with the following:

<Project Sdk="MSTest.Sdk/3.3.1">

  <PropertyGroup> 
      <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>

  <!-- Additional properties and items for your tests. -->
</Project>

Note that you can use any target framework supported by MSTest (i.e. net462 and above).

Advantages of MSTest SDK

This new SDK offers many advantages for you and your test projects, such as:

  • Better defaults
  • Simplified usage
  • Extensibility of the MSTest runner
  • Easier opt-in for new features (e.g. native AOT tests).

Better defaults

When using the MSTest SDK, you are aligning with the patterns that are provided by the main types of applications such as ASP.NET Core, Razor, Windows Desktop. It will use the default suggestions that the MSTest team makes for your test projects.

For example, we introduced some MSTest static code analyzers with v3.2 but these analyzers are defined in a new package that is not available by default so you would have to manually add this package to your test projects. Instead, by using MSTest SDK, you would simply bump the version and get all the new defaults.

We want to call out that we are following semantic versioning (see this article about semantic versioning) principles closely when choosing the defaults and updating them between versions, so you have the guarantee of understandable and easy updates.

Easier usage of MSTest Runner extensions

In addition to MSTest runner, we are also shipping a set of extensions that you can install as NuGet packages to enhance your testing experience. To help you select the right defaults, the right extensions and to make sure you have easy update and alignments between the extensions, we are introducing a new concept of “profiles”. We are currently providing the following 3 profiles: Default, AllMicrosoft and None, that you can configure and further customize. With Default profile being our recommendation.

The default profile contains:

Starting from any profile, you can then manually opt-in or opt-out any additional extension adding some additional properties to your project that would follow the pattern Enable[NugetPackageNameWithoutDots]. For example, to add crash dump support to the default profile you can simply add the following MSBuild property EnableMicrosoftTestingExtensionsCrashDump set to true.

<Project Sdk="MSTest.Sdk/3.3.1">

  <PropertyGroup>
      <TargetFramework>net8.0</TargetFramework>
      <!-- Enable Microsoft.Testing.Extensions.CrashDump package on top of the default profile -->
      <EnableMicrosoftTestingExtensionsCrashDump>true</EnableMicrosoftTestingExtensionsCrashDump>
  </PropertyGroup>

  <!-- Additional properties and items for your tests. -->
</Project>

You can refer to our MSTest SDK documentation to get more information about these profiles and their defaults.

Testing Native AOT

MSTest is the first .NET Test Framework and runner to support running tests in Native AOT mode. When using MSTest SDK, we will automatically detect if you are publishing to AOT and transparently swap all required test packages and configurations to match this specialized mode.

For more information about testing with Native AOT, please refer to this blog post: Testing with Native AOT blog post.

Example of project setup without MSTest SDK:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>

    <OutputType>exe</OutputType>
    <PublishAot>true</PublishAot>
  </PropertyGroup>

  <ItemGroup>
    <!-- 
      Experimental MSTest Engine & source generator, 
      close sourced, licensed the same as our extensions 
      with Microsoft Testing Platform Tools license.
    -->
    <PackageReference Include="MSTest.Engine" Version="1.0.0-alpha.24163.4" />
    <PackageReference Include="MSTest.SourceGeneration" Version="1.0.0-alpha.24163.4" />

    <PackageReference Include="Microsoft.CodeCoverage.MSBuild" Version="17.10.4" />
    <PackageReference Include="Microsoft.Testing.Extensions.CodeCoverage" Version="17.10.4" />

    <PackageReference Include="Microsoft.Testing.Extensions.TrxReport" Version="1.0.2" />
    <PackageReference Include="Microsoft.Testing.Platform.MSBuild" Version="1.0.2" />
    <PackageReference Include="MSTest.TestFramework" Version="3.2.2" />
    <PackageReference Include="MSTest.Analyzers" Version="3.2.2" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\ClassLibrary1\ClassLibrary1.csproj" />
  </ItemGroup>

  <ItemGroup>
    <Using Include="Microsoft.VisualStudio.TestTools.UnitTesting" />
  </ItemGroup>

</Project>

The same project setup with MSTest SDK:

<Project Sdk="MSTest.Sdk/3.3.1">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <PublishAot>true</PublishAot>
  </PropertyGroup>

  <ItemGroup>
    <ProjectReference Include="..\ClassLibrary1\ClassLibrary1.csproj" />
  </ItemGroup>

  <ItemGroup>
    <Using Include="Microsoft.VisualStudio.TestTools.UnitTesting" />
  </ItemGroup>

</Project>

What’s next?

The MSTest SDK style, while still in development, stands as the cornerstone of our forthcoming evolutions and features. We strongly encourage all MSTest users to transition to this SDK style that will become our standard for the MSTest project template with .NET 9.

We are also planning to add more scenarios in upcoming releases such as Playwright and WinUI.

We welcome any feedback on how to improve and refine it further, ensuring it aligns with the diverse needs and expectations of our users. The best way to share feedback is to creat an issue in our microsoft/testfx repository.

Marco Rossignoli Senior Software Engineer, DevDiv - .NET Developer Experience

Amaury Levé Senior Software Engineer, .NET Developer Experience

Read next

19 comments

Leave a comment

  • Daniel Smith 2

    Great work guys!

    Why doesn’t this post appear on the main DevBlogs feed though? (https://devblogs.microsoft.com/) I would have missed it entirely if it wasn’t for someone posting the direct link on Reddit.

    • Amaury LevéMicrosoft employee 0

      Hi Daniel,

      Thank you for the feedback, I have reached out to the blog team to understand what we missed to be on the devblogs feed.

  • Howard van Rooijen 1

    Great news! Are you considering adding support for testing WASM? I’m a maintainer of Reactive Extensions for .NET and we currently have open issues about bugs when used in WASM (Blazor) scenarios, and we can’t find an officially supported approach!

    Thanks!

    • Amaury LevéMicrosoft employee 0

      We definitely want to do it! We know the platform under MSTest runner already supports it but there is work on the framework engine, Visual Studio Test Explorer (and VS Code one) and most likely a good phase of testing to be sure it’s covered correctly.

      You can track this issue (I think someone from your team posted it) https://github.com/microsoft/testfx/issues/2196

  • Hanz LLanto 0

    Will MsTest replace xUnit?

    • Amaury LevéMicrosoft employee 0

      I don’t understand your question.

      MSTest SDK is a feature that we, MSTest team, ship to simplify and improve user experience of MSTest users. xUnit maintainers or any other library can also provide their own custom project SDK.

      Each user is able to chose any test framework they desire based on their interest, preference, context or available features.

  • Mayank 0

    Please can you show me more this one? it will be good if you give deep information on this one. thanks

  • James 1

    Is there an easy way to support running tests both with and without native AOT? I’m thinking the common scenario for libraries which would support both kinds of consumers.

    • Amaury LevéMicrosoft employee 0

      Hi James! Yes! MSTest SDK is the perfect fit for such use case. By simply providing the PublishAot MSBuild property, we would change all needed references for you. So you can easily do 2 calls to the test execution command (dotnet test, dotnet exec, dotnet run or calling the exe), one with and the other without the property.

  • James Wiseman 0

    I follow the Slack channel for Stryker mutation testing, and i was wondering what this might mean for such integrations.

    One of the big complaints is that the biggest bottleneck for speed is vstest, in that it it spins up a fresh completely isolated testrunner process for every test session.

  • Igor NesterovMicrosoft employee 1

    Good job!
    Does this feature work with SDK style projects targeted on net472?

    • Amaury LevéMicrosoft employee 0

      Hi Igor,

      Thank you! MSTest.Sdk is working with all supported target frameworks of MTest as long as you are using SDK style project.

  • Alex Lambert 1

    Will the nuget package manager show when there are updates to the SDK and can I update through it? Otherwise that would be a downgrade

    • Amaury LevéMicrosoft employee 0

      Hi Alex,

      That’s a good question and something we should have mentionned in the blogpost (I’ll update the doc to make it clear). There are currently some limitations (see https://github.com/NuGet/Home/issues/13127) that we hope will be addressed by the other teams.

      We have experimented/dogfooded this principle for a while and using the global.json file setup this didn’t feel much of a pain. If you are having a different experience, please do raise the point in our repo https://github.com/microsoft/testfx

Feedback usabilla icon