What’s New in Our Code Coverage Tooling?

Jakub Chocholowicz

Exciting news for developers! We’ve enhanced our code coverage tools, Microsoft.CodeCoverage and dotnet-coverage, with some fantastic features. If you’re new to our tools, check out our Get Started guide. Let’s dive into the changes that will simplify your coding experience.

Support for All Platforms

Our tools can run on any platform supported by .NET, thanks to the addition of static instrumentation. Learn more about static and dynamic instrumentation, and discover supported platforms.

Fresh Report Formats

We’ve revamped our code coverage report formats to integrate smoothly with tools like ReportGenerator. While the default remains the familiar .coverage format, we’ve introduced some new ones:

  • Binary (Default): .coverage (Microsoft’s special format) – Open it in Visual Studio Enterprise. Example
  • Cobertura: .cobertura.xml (Open-source XML format) – Open it in Visual Studio Enterprise, any text editor, or generate an HTML report with ReportGenerator. Example
  • XML: .xml (Microsoft’s XML Format) – Open it in Visual Studio Enterprise and any text editor. Example

Meet dotnet-coverage

Introducing our new tool, dotnet-coverage! It performs following tasks:

  • Collects code coverage for console applications. Example
  • Collects code coverage for web applications. Example
  • Merges coverage reports. Example
  • Instruments binaries. Example
  • Calculates code coverage for each test separately. Example

Visit dotnet-coverage documentation to learn more.

Auto-Merge for solutions

Running dotnet test --collect "Code Coverage" at the solution level now automatically merges code coverage for all your test projects. Visit Scenario 24 Code coverage for solution to see full example.

Improved Documentation

Explore our fresh GitHub repository at microsoft/codecoverage for all the info and samples you need.

Faster Performance

Prior to the 16.5 release, the collection of code coverage report significantly slowed down test execution. We addressed this issue, resulting in an impressive 80% performance gain. See performance section for detailed results and logs.

Package Time Ratio
Microsoft.CodeCoverage 16.5 03:52:53 1.00
Microsoft.CodeCoverage 17.0 02:25:49 0.63
Microsoft.CodeCoverage 17.5 01:27:52 0.38
Microsoft.CodeCoverage 17.9 00:50:00 0.21

What You Need to Do

To enjoy the latest features and speed up your builds, make sure to use our latest stable packages in your test projects:

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.8.0" />
<PackageReference Include="Microsoft.CodeCoverage" Version="17.8.0" />

If your solution doesn’t have any C++ code, make it faster and more reliable by turning off native code coverage with these flags in runsettings:

<EnableStaticNativeInstrumentation>False</EnableStaticNativeInstrumentation>
<EnableDynamicNativeInstrumentation>False</EnableDynamicNativeInstrumentation>

Visit configuration documenation to see other options and full example of our settings.

Special Thanks

A big thank you to Faisal Hafeez, Marco Rossignoli, Mariam Abdullah, Codrin-Victor Poienaru and Pavel Horak for their exceptional contributions to this project! 🙌🚀

19 comments

Comments are closed. Login to edit/delete your existing comments

  • silkfire 5

    How does Microsoft.CodeCoverage compare to Coverlet?

    • Chris DaMour 1

      came here to ask this same thing. an analysis from MS perspective would be very valuable

    • Jakub ChocholowiczMicrosoft employee 2

      If

      1) you don’t need to know how many times each line was executed
      2) it’s not mandatory for you to use only open-source tools in your repository
      3) you don’t need to get coverage in a constrained environment like the case of System.Private.Corelib.dll where there is no support of BCL

      we recommend switching to Microsoft.CodeCoverage for better performance and reliability.

      Please check below links to see performance comparison between Coverlet and Microsoft.CodeCoverage:
      https://github.com/dotnet/roslyn-analyzers/pull/6211#issuecomment-1403679164
      https://github.com/dotnet/roslyn-analyzers/pull/6211#issuecomment-1409894782

  • Paulo Morgado 2

    Will this replace coverlet in the unit test templates?

    • Jakub ChocholowiczMicrosoft employee 1

      Microsoft.CodeCoverage is a dependency of Microsoft.NET.Test.Sdk. It means that both tools Microsoft.CodeCoverage and Coverlet are installed by unit test templates. After generating new project you can execute:

      dotnet test --collect "Code Coverage"

      to run tests under Microsoft.CodeCoverage or:

      dotnet test --collect "XPlat Code Coverage"

      to run tests under Coverlet.

      • Paulo Morgado 1

        Thanks Jakub,

        My question was more if templates dotnet new mstest and the others will be using Microsoft.CodeCoverage instead of Coverlet.

  • Michael Dietrich 0

    This looks really great.
    But something that confuses me a lot is when do I need dotnet-coverage collect and when is dotnet test --collect enough?
    In some samples both are used in others not. Unfortunately it’s not clear to me.

    And since Microsoft.CodeCoverage is a dependency of Microsoft.NET.Test.Sdk, there is no need to explicitly reference Microsoft.CodeCoverage for test assemblies, since it is already referenced transitively?

    • Jakub ChocholowiczMicrosoft employee 3

      dotnet-coverage collect collects code coverage for any managed tree of processes (ASP.NET server, console application). dotnet test --collect is specific for test projects and runs tests with code coverage collector enabled. Generally using dotnet test --collect is enough in most scenarios and it is recommended.

      If your test project communicates with external service which is started before test execution, then to get code coverage for external service you have to use dotnet-coverage tool. This scenario you can find here: https://github.com/microsoft/codecoverage/blob/main/samples/Calculator/scenarios/scenario14/README.md

      I understand your confusion here as running tests (dotnet test) is also managed process so you can do: dotnet-coverage collect "dotnet test" and the result is equivalent to dotnet test --collect "Code Coverage". Command dotnet-coverage collect "dotnet test" can be used as workaround for issue: https://github.com/microsoft/vstest/issues/2378 and it is documented here: https://github.com/microsoft/codecoverage/blob/main/samples/Calculator/scenarios/scenario25/README.md

      You are right regarding dependencies. It is enough to have in csproj Microsoft.NET.Test.Sdk. Our recommendation is to keep it up to date as we are constantly improving code coverage.

      • Christian Andritzky 0

        What happens when you run

        dotnet test --collect "Code Coverage"

        and the unit test project starts (and gracefully terminates) the server (as a child process)? Will Microsoft.CodeCoverage collect the code coverage of the server (child) process? coverlet does this if (and only if) the server (child) process is properly terminated.

  • Michael Dietrich 2

    I realized the resulting code coverage reports also contain code coverage information for external code such as MassTransit.
    Is there an easy way to only collect code coverage information for code files that actually exist locally? Since depending on the referenced NuGets, this can fairly distort the resulting code coverage report. E.g. in the mentioned case I can see several entries like /_/src/MassTransit.Abstractions/Attributes/ConfigureConsumeTopologyAttribute.cs which we would like to filter out.

  • Christian Andritzky 0

    “Auto-Merge for solutions” is a great and very welcome addition. But will it work when there are test projects with multi-targeting running tests e.g. for .NET Framework and modern .NET? We use this a lot while we are migrating a huge application from .NET Framework to .NET 6 (or 8). AFAIK you cannot execute .NET Framework tests and .NET (Core) tests within the same test host process. You need two separate runs/processes for this.

    How does this work with the new CodeCoverage features? Is there a sample for this scenario?

    • Jakub ChocholowiczMicrosoft employee 1

      Yes, this is supported. You are right that each test project is running on specific test host process. After all projects are done code coverage reports for all projects are merged together. We don’t have sample for this.

  • Jeff Chen 0

    This is a great update. I am trying to use this package in my azure pipeline. The question is that how to spefiic ‘Format’ in dotnet test task via a command line ?

    When I try this in ‘Arugments’ field of dotnet test task:

    --no-build --collect "Code Coverage;Format=cobertura;"

    the task will show following message:

    Data collection : Unable to find a datacollector with friendly name 'Format=cobertura'.

    However, in the document here:
    https://github.com/microsoft/codecoverage/blob/main/README.md
    It shows a sample code like this:

    dotnet test --collect "Code Coverage;Format=cobertura"

    So, it looks like the approach used in command line cannot be used in dotnet test task ?

    What is the correct way to use ‘Format’ parameter in this task? thanks a lot

    • Jakub ChocholowiczMicrosoft employee 0

      What version of .NET SDK you are using? I think updating it will help.

Feedback usabilla icon