In an earlier post we detailed the fresh approach we have taken with unit testing in Visual Studio 11. You can now pick and choose the test framework of your choice and use it from within the Visual Studio experience in a fully integrated manner. While we have done a lot of work on the new experience and a new extensible platform, we have also invested in the built-in MSTest framework for unit-testing. In this post, I wanted to highlight the investments we have done in MSTest.

Performance

Performance is a central theme across Visual Studio 11 and developer unit testing scenarios with MSTest have also received a performance boost. Our internal tests indicate that we are significantly faster compared to VS 2010 and comparable to other popular unit test frameworks.

MSTest in VS 2010 is a monolithic general purpose test framework for unit tests, Coded UI Test and Load and Web Performance test and supports advanced scenarios like remote execution, data collectors, custom testsettings. While these features are very useful for many testing scenarios, we observed that they are not always relevant for developer unit tests but were contributing to startup/test execution time. In VS 11, we built a new, lightweight, faster, no frills test execution adapter for MSTest unit tests. It only supports unit-tests and does not support advanced scenarios like testsettings. This allowed us to achieve the performance enhancements for developer unit tests scenarios.

In addition to the above lightweight adapter, we also have an adapter for classic the MSTest framework which supports other test types like Coded UI Tests and existing unit/integration tests that use advanced features like testsettings.

Simplified & Separate Unit Test Project

In line with the above approach, we have also simplified project creation for unit tests and now provide separate project template optimized for unit tests that does not include assets like testsettings file by default. Creating an MSTest unit test project should be a much faster, cleaner experience in Visual Studio 11. Similarly, we now have separate project template for Coded UI Tests and Load tests.

Compatibility with VS 2010

Another theme for this release is compatibility between projects created in Visual Studio 2010 and Visual Studio 11. Developers should be able to open their existing VS 2010 test projects in Visual Studio 11 without requiring any upgrade. Similarly, they should be able to move back to VS 2010 and continue working on them.

The adapter for classic MSTest framework also provides compatibility and support for testsettings file that might be in use in a Visual Studio 2010 project.

Introducing Express SKU Support

With Visual Studio 11, MSTest unit test framework makes a debut in the Express SKUs. It is now available as part of Visual Studio 11 Express for Web and Visual Studio 11 Express for Windows 8. Developers can now use Express for Web to author, run, and debug MSTest based unit tests for their MVC and Azure projects. They can also queue tests to run as part of a TFS build and view the test results from the build report. We have also enabled first class unit testing support for Windows 8 Metro Style apps in Visual Studio 11 Express for Windows 8.

Support for unit testing Windows 8 Metro style apps

Windows 8 is a reimagining of Windows, where apps take center stage. Visual Studio 11 Express for Windows 8 provides great tools for developing, debugging and testing Metro style apps. The familiar MSTest framework is available to developers to write unit tests to test out their XAML based Metro style apps and WinRT components. Special project templates for unit tests are available. Developers can specify capabilities that are needed for their tests and app. The tests get executed in an updated Windows 8 compatible test execution process.

Support for Async Tests

The Windows Runtime, in Windows 8, and .Net Framework 4.5 allows developers to build responsive applications using the new asynchronous programming model. MSTest framework has been updated to support testing of asynchronous code/apis. For example, a developer can create an asynchronous API that needs to be tested. By leveraging the async/await pattern and .Net Tasks, the developer can quickly write an async unit-test as follows:

 [TestMethod] 
 public async Task AsyncTest() 
 
 { 
 
 var result = await MyOperationAsync(); 
 
 Assert.IsTrue(result); 
 
 } 
 

MSTest framework in Beta provides special handling for async task returning tests and simplifies testing of asynchronous apis.

64 Bit Support and .Net Multi-targeting Support

In VS 2010, unit tests could be run as 64 bit only if it and its dependencies were “any cpu”. Visual Studio 11 has been updated to remove that limitation. Tests can reference 64 bit product code now. Similarly, .Net multi-targeting for unit tests is built from ground up. We support .Net Framework 3.5 and above. Both of the above features are critical when developing tests for server components such as SharePoint.

I hope this is a useful set of enhancements. But we are not done yet. We are aware of areas where MSTest comes under fire for lagging behind. Missing features like better Assert libraries, simpler extension model, RowTest! etc, have been common asks and have inspired passionate debates. These enhancements are on our radar.

In summary, Visual Studio 11 provides an enhanced test framework in MSTest in the box. At the same time it embraces extensibility and works equally well with any test framework, enabling developers to be in complete control and enjoy a great development experience.

Abhishek Agrawal,

Senior Program Manager, Visual Studio ALM

Microsoft