As promised in the previous post, what follows is a recap of all of the Testing-related features implemented and delivered in the Visual Studio 2015 cycle until now. Each of the features might be relevant at a different stage in the lifecycle, but together they serve the single purpose of enabling efficiency – and overlaid on the lifecycle graphic, that constellation emerges.

Very briefly, the features may be described as follows:

• Invigorated, “MSTest V2” (4) comes with long awaited feature-additions, and joins the list of third party test frameworks and adapters available on NuGet (1), enabling it to have its own release/update cadence, and making easy to search for, install, uninstall, and update it as a dependency to the code-under-test.
• The Create Unit Tests (2) wizard enables quick bootstrapping of tests while in the flow of coding by making it easy to configure a test project, and the test class and the test method stub therein.
• But we know that developing a suite of unit tests that exhaustively exercise and validate the logic of the code under test is a challenge – and even more so when the underlying code-under-test is constantly evolving. IntelliTest (3) helps overcome this daunting challenge, by automatically generating a suite of tests, and automatically keeping them updated.
• Incremental validation (5) enables fast turnaround in the red-green-refactor “dev inner-loop”, and does so by selecting tests only from test containers touched by the build – the Test Explorer window’s “Run Test After Build” control then runs only that selection.
• In Visual Studio 2015 Update 3, test execution is faster (9) even if you don’t use parallel execution, and memory consumption is lower (6) during both discovery and execution of tests. These improvements are pronounced for large solutions containing thousands of tests spread over many test containers.
• Assembly Resolution (8) provides a convenient means to configure the locations from where to load the dependencies for a test during its execution – especially useful when you want to run tests in a lab environment against an installed product (i.e. when explicitly placing dependent DLLs next to the test DLL might be impractical).
• Parallel Test Execution (10) is supported for all test adapters, using the test container (i.e. the file containing the tests to be executed) as the unit of parallelization, and a process as the unit of isolation. Within each container, the tests will be executed as per the semantics of the corresponding test framework. It can be conveniently enabled directly from the Test Explorer window, but additionally, the level of parallelism can be configured (7) through the .runsettings file.

Visual Studio “15” Preview There has been a lot of feedback on the Test Explorer. We have begun addressing this feedback from Visual Studio “15” Preview 2 onwards.

• Group by namespace (11) has been added, and completes the complement of groupings available through the Test Explorer.
• On a build failure, the option to run tests from a prior build (12) can now be controlled from Tools | Options | Project and Solutions | Build and Run hive.
• The Test Explorer shows up automatically (13) when you run tests making it easy to know what tests are running.
• The new h:mm:ss:sss format is used to report test duration precisely (14).
• The test duration is reported uniformly (15) on each test, in the Summary pane, in the Output window, and in the test report seen in CI.
• Furthermore, when grouped by duration, the tests within each group are sorted (16) such that the slower running tests do not hide, but instead float to the top.

Perusing through an output log file would abruptly end with the message “!!LOG TRUCATED!! To get a complete log select “Copy All” and paste into some text editor like Notepad”. Quickly wanting to look up say, an exception message in code was no easier – since select-and-copy was not supported in the Test Window. That is changed with Visual Studio “15”:

• Select-and-copy is supported (17) in the Summary pane and in the Test Output window.
• The logs are no longer truncated (18).

Summary There, 18 features, with a common purpose: efficiency in getting from conceptualizing the tests, all the way to the point where they are generating reports that can inform business decisions.

Each of these features merit a post of its own. But that will be for another day. In the meantime leave us your feedback. We are listening.

Next post – next steps for the vstest platform Where is the test platform headed? What are the next steps in that direction?

Stay tuned.