{"id":9771,"date":"2015-07-30T05:37:00","date_gmt":"2015-07-30T05:37:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/visualstudioalm\/2015\/07\/30\/speeding-up-unit-test-execution-in-tfs\/"},"modified":"2022-08-02T00:50:07","modified_gmt":"2022-08-02T08:50:07","slug":"speeding-up-unit-test-execution-in-tfs","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/devops\/speeding-up-unit-test-execution-in-tfs\/","title":{"rendered":"Speeding up Unit Test Execution in TFS"},"content":{"rendered":"<p>Users of TFS can do a few simple things to speed up Unit Test execution and improve the overall Build time.<\/p>\n<p><strong>Maximize hardware resource utilization<\/strong><\/p>\n<p>In most likelihood, you run your tests run on a hardware with multiple cores. To make best use of all the cores, you can use the ability of test frameworks to create multiple threads and run your tests in parallel. Multiple test frameworks like XUnit, TestNG, etc support multi-threaded execution. As an example, to enable threads in TestNG, the number of threads are defined in an xml file like below.<\/p>\n<p style=\"text-align: left\">\n  <a href=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/6283.i1.jpg\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/6283.i1.jpg\" alt=\"\" border=\"0\" \/><\/a>\u00a0\n<\/p>\n<p style=\"text-align: left;padding-left: 240px\">\n  Defining number of threads in TestNG\n<\/p>\n<p align=\"center\">\n  \u00a0\n<\/p>\n<p>Parallelism can be enabled at different levels like test cases, classes, etc and depends on how thread safe your tests are. Depending on the degree of parallelism, you can easily see more than 50% reduction in test run time.<\/p>\n<p><strong>Distribute runs to multiple machines<\/strong><\/p>\n<p>Another way to speed up your unit tests is to distribute your test runs to multiple machines. For example, if you have 3 Build Agent machines, you could distribute 1\/3<sup>rd<\/sup> of the tests to each machine and achieve a faster build time.<\/p>\n<p>Lets explore how we can use this technique in TFS. We will start with a Build Definition that build and then runs unit tests in a serial manner, like the one below.<\/p>\n<p style=\"text-align: left\">\n  \u00a0<a href=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/3731.i2.jpg\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/3731.i2.jpg\" alt=\"\" border=\"0\" \/><\/a>\n<\/p>\n<p>\u00a0<\/p>\n<p>Step1. Divide your unit tests into logical sets. For example, you can divide your tests equally across the sets. Or you can do it in terms of priority, or any other way that works for you. For this example, I divide my tests into 3 sets \u2013 MyShuttleTests, MyShuttleClientTests and MyShuttleServerTests<\/p>\n<p>\u00a0<\/p>\n<p>Step2. Add a new variable in the Build Definition with values MyShuttleTests, MyShuttleClientTests and MyShuttleServerTests. I name the variable MyShuttle<\/p>\n<p style=\"text-align: left\">\n  \u00a0<a href=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/2146.i3.jpg\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/2146.i3.jpg\" alt=\"\" border=\"0\" \/><\/a>\n<\/p>\n<p style=\"text-align: left\">\n  Step3. In the Options section, check the box for \u2018MultiConfiguration\u2019, use the variable name \u2018MyShuttle\u2019 in the Multipliers textbox and also check Parallel.\n<\/p>\n<p style=\"text-align: left\">\n  \u00a0<a href=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/4747.i4.jpg\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/4747.i4.jpg\" alt=\"\" border=\"0\" \/><\/a>\n<\/p>\n<p>\u00a0<\/p>\n<p>Step4. Use the variable name in the Test selection \/ filtering logic of the test task so that one of the slices gets picked up.<\/p>\n<p style=\"text-align: left\">\n  \u00a0<a href=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/4380.i5.jpg\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/4380.i5.jpg\" alt=\"\" border=\"0\" \/><\/a>\n<\/p>\n<p>\u00a0<\/p>\n<p>Step5. Save and Queue the Build. When the Build starts running, 3 parallel builds are created for the 3 values of the \u2018Multiplier\u2019 variable (MyShuttle). Since \u201cParallel\u201d was checked, all 3 builds will start at the same time provided 3 Build Agent machines are available in the Build pool.<\/p>\n<p>Note the 3 builds with the 3 values of the variable MyShuttle starting in parallel.<\/p>\n<p style=\"text-align: left\">\n  \u00a0<a href=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/8623.i6.jpg\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/8623.i6.jpg\" alt=\"\" border=\"0\" \/><\/a>\n<\/p>\n<p>\u00a0<\/p>\n<p>\u00a0Once the run is complete, you can see your test results as in the screenshot below. Since we divided up the tests into 3 sets, we get 3 different runs reported against the build.<\/p>\n<p style=\"text-align: left\">\n  \u00a0<a href=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/3348.i7.jpg\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/3348.i7.jpg\" alt=\"\" border=\"0\" \/><\/a>\n<\/p>\n<p>\u00a0<\/p>\n<p>This approach works for non C# unit tests also. For example, for JAVA tests running using an Ant task, you can create multiple build.xml files, each of which points to one of the test suites. These files will then become the \u2018Multiplier\u2019 for the parallel runs. Below screenshots show sequence of steps for an ANT task.<\/p>\n<p style=\"text-align: left\">\n  \u00a0<a href=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/5050.i8.jpg\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/5050.i8.jpg\" alt=\"\" border=\"0\" \/><\/a>\n<\/p>\n<p style=\"text-align: left;padding-left: 240px\" align=\"center\">\n  Defining the variable with build files\n<\/p>\n<p align=\"center\">\n  \u00a0\n<\/p>\n<p style=\"text-align: left\">\n  \u00a0<a href=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/2677.i9.jpg\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/2677.i9.jpg\" alt=\"\" border=\"0\" \/><\/a>\n<\/p>\n<p style=\"text-align: left;padding-left: 240px\" align=\"center\">\n  Using the variable in the ANT task\n<\/p>\n<p align=\"center\">\n  \u00a0\n<\/p>\n<p style=\"text-align: left\">\n  \u00a0<a href=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/8032.i10.jpg\"><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2015\/07\/8032.i10.jpg\" alt=\"\" border=\"0\" \/><\/a>\n<\/p>\n<p style=\"text-align: left;padding-left: 240px\">\n  <span style=\"text-align: center\">Test results page<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Users of TFS can do a few simple things to speed up Unit Test execution and improve the overall Build time. Maximize hardware resource utilization In most likelihood, you run your tests run on a hardware with multiple cores. To make best use of all the cores, you can use the ability of test frameworks [&hellip;]<\/p>\n","protected":false},"author":176,"featured_media":45953,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-9771","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops"],"acf":[],"blog_post_summary":"<p>Users of TFS can do a few simple things to speed up Unit Test execution and improve the overall Build time. Maximize hardware resource utilization In most likelihood, you run your tests run on a hardware with multiple cores. To make best use of all the cores, you can use the ability of test frameworks [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts\/9771","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/users\/176"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/comments?post=9771"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts\/9771\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/media\/45953"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/media?parent=9771"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/categories?post=9771"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/tags?post=9771"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}