Speeding up Unit Test Execution in TFS

Atin Bansal

July 30th, 20150 0

Users of TFS can do a few simple things to speed up Unit Test execution and improve the overall Build time.

Maximize hardware resource utilization

In most likelihood, you run your tests run on a hardware with multiple cores. To make best use of all the cores, you can use the ability of test frameworks to create multiple threads and run your tests in parallel. Multiple test frameworks like XUnit, TestNG, etc support multi-threaded execution. As an example, to enable threads in TestNG, the number of threads are defined in an xml file like below.

Defining number of threads in TestNG

Parallelism can be enabled at different levels like test cases, classes, etc and depends on how thread safe your tests are. Depending on the degree of parallelism, you can easily see more than 50% reduction in test run time.

Distribute runs to multiple machines

Another way to speed up your unit tests is to distribute your test runs to multiple machines. For example, if you have 3 Build Agent machines, you could distribute 1/3^rd of the tests to each machine and achieve a faster build time.

Lets explore how we can use this technique in TFS. We will start with a Build Definition that build and then runs unit tests in a serial manner, like the one below.

Step1. Divide your unit tests into logical sets. For example, you can divide your tests equally across the sets. Or you can do it in terms of priority, or any other way that works for you. For this example, I divide my tests into 3 sets – MyShuttleTests, MyShuttleClientTests and MyShuttleServerTests

Step2. Add a new variable in the Build Definition with values MyShuttleTests, MyShuttleClientTests and MyShuttleServerTests. I name the variable MyShuttle

Step3. In the Options section, check the box for ‘MultiConfiguration’, use the variable name ‘MyShuttle’ in the Multipliers textbox and also check Parallel.

Step4. Use the variable name in the Test selection / filtering logic of the test task so that one of the slices gets picked up.

Step5. Save and Queue the Build. When the Build starts running, 3 parallel builds are created for the 3 values of the ‘Multiplier’ variable (MyShuttle). Since “Parallel” was checked, all 3 builds will start at the same time provided 3 Build Agent machines are available in the Build pool.

Note the 3 builds with the 3 values of the variable MyShuttle starting in parallel.

Once the run is complete, you can see your test results as in the screenshot below. Since we divided up the tests into 3 sets, we get 3 different runs reported against the build.

This approach works for non C# unit tests also. For example, for JAVA tests running using an Ant task, you can create multiple build.xml files, each of which points to one of the test suites. These files will then become the ‘Multiplier’ for the parallel runs. Below screenshots show sequence of steps for an ANT task.