{"id":2003,"date":"2010-06-01T13:47:42","date_gmt":"2010-06-01T13:47:42","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/visualstudio\/2010\/06\/01\/better-parallelism-in-msbuild-4-with-yieldduringtoolexecution\/"},"modified":"2022-10-13T12:57:48","modified_gmt":"2022-10-13T19:57:48","slug":"better-parallelism-in-msbuild-4-with-yieldduringtoolexecution","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/visualstudio\/better-parallelism-in-msbuild-4-with-yieldduringtoolexecution\/","title":{"rendered":"Better Parallelism in MSBuild 4 with YieldDuringToolExecution"},"content":{"rendered":"

<\/h2>\n

<\/h2>\n

<\/h2>\n

Introduction<\/h3>\n

In MSBuild 4 we introduced several performance improvements, particular for large interdependent builds.\u00a0 By and large they are automatic and you receive their benefit without making any changes to the way your build process in authored.\u00a0 However, there are still some cases where we are unable to make the best decision.\u00a0 One such case is when there is a particular external tool which is invoked as part of the build but which takes a significant amount of time.\u00a0 An example of such a tool would be cl.exe, the C++ compiler.\u00a0 This article discusses how to use the new yield mechanism for external tools to improve the performance of your builds.<\/p>\n

Tool Tasks<\/h3>\n

There are a few ways MSBuild can be made to execute external, command-line tools:<\/p>\n

    \n
  1. Write a task which derives from ToolTask.<\/li>\n
  2. Use the Exec task to call your command.<\/li>\n
  3. Use the XamlTaskFactory.<\/li>\n<\/ol>\n

    All of these methods ultimately use the ToolTask class in Microsoft.Build.Utilities.v4.0.dll to handle executing a command-line task and deal with the output in the MSBuild way.\u00a0 Like all tasks, however, they block any other work from happening in MSBuild while they are executing.\u00a0 In cases where the task is very short, such as touching a log file or copying a file from one place to another this is perfectly acceptable.\u00a0 But in the original example of invoking the C++ compiler, the amount of time MSBuild itself sits idle can be lengthy and in some cases it may be a significant impediment to good parallelization of your build.<\/p>\n

    The problem has to do with the way MSBuild utilizes its worker nodes.\u00a0 Whenever a project is scheduled to be built, it is assigned to one of the worker nodes.\u00a0 This node will then execute that project from start to finish, and will not accept more work until the project is either finished or the project makes an MSBuild call (for instance to satisfy a project-to-project reference.)\u00a0 This is in large part because a node can only execute one task at a time, as tasks must be guaranteed their environment and current directory will not be modified during execution.<\/p>\n

    However, command-line tools do not execute in-process, and therefore their environment cannot be polluted by the running of additional tasks in parallel on the same node.\u00a0 We can take advantage of this behavior to let the MSBuild node execute tasks in other projects while our long-running tool completes its work.\u00a0 This is done using the YieldDuringToolExecution parameter.<\/p>\n

    YieldDuringToolExecution<\/h3>\n

    In order to allow MSBuild to continue building other projects while a command-line tool in one project is running is simple.\u00a0 Just set the YieldDuringToolExecution parameter to \u2018True\u2019 on your long running command-line tool.\u00a0 This is a boolean parameter, so any valid MSBuild expression which resolves to a boolean value will work.\u00a0 Here\u2019s an example:<\/p>\n\n\n\n
    \n
    <PropertyGroup>\r\n    <YieldDuringToolExecution>true<\/YieldDuringToolExecution>\r\n<\/PropertyGroup>\r\n<\/span>\r\n<Exec CommandLine=\u201dSleep 10000\u201d YieldDuringToolExecution=\u201d$(YieldDuringToolExecution)\u201d\/><\/span><\/pre>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
    When the Exec task executes, normally it would sleep for 10000 seconds during which no other work on the node can proceed.\u00a0 However, with yielding enabled, the Sleep command will still run but the MSBuild node will be free to do other work.\u00a0 Once the Sleep command is finished, the node will resume building the project which launched it as soon as the node is free to do so.<\/p>\n
    Whether or not you should enable yielding for your ToolTasks depends on what they do.\u00a0 Generally speaking if the task runs for less than one second, it\u2019s probably not worth it to enable this since there is a small cost to give up the MSBuild node.\u00a0 However, for longer tools you may see some wins, and the wins will likely be larger the more complex your build is and the more long running tasks you have in it.\u00a0 Again, large interdependent C++ builds are a great example of this and they benefit tremendously from yielding being applied to the compiler.\u00a0 You can investigate your build\u2019s performance using the Detailed Summary<\/a> feature of MSBuild 4.<\/p>\n
    Yielding interacts well with the \/m switch in MSBuild as well.\u00a0 For instance, if you have specified \/m:4 to enable parallelization, MSBuild will ensure that no more than four parallel things are going on at once, whether they be regularly building projects or yielding tools.\u00a0 So enabling yielding will not cause your machine to become more overloaded.\u00a0 Instead your builds are likely to improve their parallelization and make better use of available CPU and I\/O cycles that they would otherwise.<\/p>\n
    We have already enabled yield semantic for several tool tasks.\u00a0 These include:<\/p>\n