{"id":2363,"date":"2010-03-07T21:12:00","date_gmt":"2010-03-07T21:12:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/visualstudio\/2010\/03\/07\/tuning-c-build-parallelism-in-vs2010\/"},"modified":"2019-02-14T15:39:43","modified_gmt":"2019-02-14T23:39:43","slug":"tuning-c-build-parallelism-in-vs2010","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/visualstudio\/tuning-c-build-parallelism-in-vs2010\/","title":{"rendered":"Tuning C++ build parallelism in VS2010"},"content":{"rendered":"<p>A great way to get fast builds on a multiprocessor computer is to take advantage of as much parallelism in your build as possible. If you have C++ projects, there\u2019s two different kinds of parallelism you can configure. <\/p>\n<h3>What are the dials I can set?<\/h3>\n<p>Project-level parallel build, which is controlled by MSBuild, is set at the solution level in Visual Studio. (Visual Studio actually stores the value per-user per-computer, which may not be always what you want \u2013 you may want to have different values for different solutions, and the UI doesn\u2019t allow you to do that.). By default Visual Studio picks the number of processors on your machine. Do some experiments with slightly higher and lower numbers to see what gives the best speed for your particular code. Some people like to dial it down a little so that they can do other work while a build goes on.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_14.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_6.png\" width=\"706\" height=\"411\"><\/a> <\/p>\n<p>This dial is just the same as VS2008, although under the covers MSBuild is taking over some of the work from Visual Studio now.<\/p>\n<p>If you\u2019re building C++ or C++\/CLI, there\u2019s another place you can get build parallelism. The CL compiler supports the \/MP switch, which tells it to build subsets of its inputs concurrently with separate instances of itself. The default number of buckets, again, is the number of CPU\u2019s, but you can also specify a number, like \/MP5. Again, this was available before, so I\u2019m going to just remind you where the value is and what it looks like in the MSBuild format project file.<\/p>\n<p>Go to your project\u2019s property pages, and to the C\/C++, General page. For now I suggest that you select All Configurations and All Platforms. You can be more selective later if you want.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_6.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_2-1.png\" width=\"653\" height=\"464\"><\/a>&nbsp; <\/p>\n<p>As usual you can see what\u2019s in the project file by unloading it, right clicking on the node in the Solution Explorer, and choosing Edit:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_10.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_4-2.png\" width=\"440\" height=\"237\"><\/a> <\/p>\n<p>Here\u2019s what it looks like in the project file. Yes, it\u2019s inside a configuration and platform specific block, but it put the same value in all of them.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_4.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_1-3.png\" width=\"814\" height=\"143\"><\/a> <\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>Notice that it\u2019s in an \u201cItemDefinitionGroup\u201d. That MSBuild tag simply indicates it defines a&nbsp; \u201ctemplate\u201d for items of a particular type. In this case, all items of type \u201cClCompile\u201d will automatically have metadata MultiProcessorCompilation with value true unless they explicitly choose a different value.<\/p>\n<p>By the way, MSBuild Items, in case you\u2019re wondering, are just files, usually. Their subelements, if any, are the metadata. Here\u2019s what some look like. Notice they\u2019re in an \u201cItemGroup\u201d:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_12.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_5.png\" width=\"400\" height=\"125\"><\/a> <\/p>\n<p>Because this is metadata, at an extreme, I could actually set this down to a per-file basis. In that case, MSBuild would bucket together all the inputs that have a common value. You would need to disable \/MP for particular files that use #import, for example, because that&#8217;s not supported with \/MP. (Other features not supported with \/MP are \/Gm, which is incremental compilation, and a few other switches documented <a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/bb385193.aspx\" target=\"_blank\">here<\/a>)<\/p>\n<p>Note it\u2019s under the \u201cItemGroup\u201d because these are actual items:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_20.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_9.png\" width=\"579\" height=\"97\"><\/a> <\/p>\n<p>Back to multiprocessor CL. If you want to tell CL explicitly how many parallel compiles to do, Visual Studio lets you do this \u2013 as for \/MP, it&#8217;s exposed as a global setting:<a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_43.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_7.png\" width=\"589\" height=\"348\"><\/a> <\/p>\n<p>Under the covers, VS passes this on by setting a property (a global property \u2013 it&#8217;s not persisted) named CL_MPCount. That means it won&#8217;t have any effect when building outside of VS.<\/p>\n<p>If you want to choose a value at a finer grained level you can\u2019t use the UI as it\u2019s not exposed in the property pages or the command line preview. You have to go into the project file editor and type it. It\u2019s a different piece of metadata on the CLCompile items, named \u201cProcessorNumber\u201d. It can have a number from 1 to as high as you like and adds the numeric value to \/MP if you want it. If you don&#8217;t have &lt;MultiProcessorCompilation&gt; it will be ignored.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_18.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_8.png\" width=\"626\" height=\"84\"><\/a> <\/p>\n<p>The squiggle here is a minor bug \u2013 ignore it.<\/p>\n<h3>What about building on the command line?<\/h3>\n<p>The \/MP settings come from the project files, so they work exactly the same on the command line. That\u2019s part of the whole point of MSBuild, right, the same build on the command line as in Visual Studio? But the global parallelism setting that you set in Tools, Options does not affect the command line. You must pass it yourself to the msbuild.exe command with the \/m switch. Again, the value is optional and if you don\u2019t supply a value it uses the number of CPU\u2019s. However, unlike Visual Studio, out of the box, without \/m supplied, it uses 1 CPU. That might change in future.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_22.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_10-1.png\" width=\"338\" height=\"47\"><\/a> <\/p>\n<p>To choose the number on any \/MP value, you can set an environment variable, or pass a property, named CL_MPCount, just like Visual Studio does.<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<h3>Setting \/MP on every project is tiresome, what are my options?<\/h3>\n<p>Probably you\u2019ll want to use \/MP on more than one of your projects, and you don\u2019t want to edit each individually. The Visual Studio solution to this kind of problem is property sheets. They don\u2019t have any special connection to multiprocessor build, but it\u2019s an opportunity for me to give a quick refresher using this as an example. First open the \u201cProperty Manager\u201d from the view menu. Its exact location will vary depending on the settings you\u2019re using, here\u2019s where it is if you have C++ settings;<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_26.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_12.png\" width=\"252\" height=\"430\"><\/a> <\/p>\n<p>Right click on a project and choose \u201cAdd New Property Sheet\u201d:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_28.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_13-1.png\" width=\"358\" height=\"191\"><\/a> <\/p>\n<p>I have mine the name \u201cMultiprocCpp.props\u201d. You\u2019ll see it gets added to all configurations of this project. Right click on it, and you\u2019ll see the same property pages that the project has, but this time you\u2019re editing the property sheet. Again, set \u201cMulti-processor Compilation\u201d to \u201cYes\u201d. Close the property pages, select the property sheet in the Property Manager, and hit Save. <\/p>\n<p>Now I can open up that new MultiprocCpp.props file in the editor, and I see this:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_30.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_14-1.png\" width=\"595\" height=\"215\"><\/a> <\/p>\n<p>(Again, ignore the squiggle.)<\/p>\n<p>Looking in the project file, you can see the property sheet pulled in to each configuration, using an \u201cImport\u201d tag. Think of that just like a #include in C++: <\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_34.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_16-1.png\" width=\"624\" height=\"71\"><\/a> <\/p>\n<p>So now we have the definition we put in the project file before, but in a reusable form. Given that, I can put it into all the projects I want in one shot, by multi-selecting in the Property Manager and choosing Add Existing Property Sheet:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_32.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_15.png\" width=\"484\" height=\"224\"><\/a> <\/p>\n<p>Now all your projects compile with \/MP !<\/p>\n<p>In some circumstances, you might want to go beyond what you can easily do in the Property Manager. For example you might want to bulk-remove a property sheet, or put a property sheet in each project once outside of all the configurations. Fortunately MSBuild 4.0 has a powerful and complete object model over its files that you can use to do this kind of work in a few lines of code. More on that in a future blog post, but for now, if you want to take a look, point the Object Browser at Microsoft.Build.dll.<\/p>\n<p>Before I leave property sheets, it\u2019s worth mentioning that you can do this kind of common-importing in your own ways, if you don\u2019t mind losing some of the UI support. For example, in the build of VS itself, we pull in a common set of properties at the top of every project, like this example from the project that builds msenv.dll (which contains much of the VS shell)<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_36.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_17-1.png\" width=\"583\" height=\"88\"><\/a> <\/p>\n<p>Within <em>that <\/em>we define all kinds of global settings, and import yet others. I\u2019ll talk about this kind of structure in a future blog post about the organization of large build trees.<\/p>\n<h3>Too much of a good thing<\/h3>\n<p>Usually the problem is getting enough parallelism to exploit all your machine\u2019s cores. But the reverse problem is possible, and although it\u2019s a nice problem to have, it needs fixing because it will cause your machine to thrash. Here\u2019s what task manager might look like when this is happening:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_40.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_19-1.png\" width=\"338\" height=\"464\"><\/a> <\/p>\n<p>In this case on a box with 8 CPU\u2019s I enabled \/MP on all my projects in the solution, and then built it with msbuild.exe \/m (I didn\u2019t need to use the command line to have this problem, the same could happen in Visual Studio). If dependencies don\u2019t prevent it, MSBuild will kick off 8 projects at once, and in each of those CL will run 8 instances of itself at once, so we could have up to 64 copies of CL all fighting over my cores and my disk. Not a recipe for performance. <\/p>\n<p>You can expect that one day the system will auto-tune itself here, but for now if you have this problem you would do some manual adjustment. Here\u2019s some ideas:<\/p>\n<h5>Dial down the values globally<\/h5>\n<p>Reduce \/m:4 to \/m:3, for example, or use a property sheet to change \/MP to \/MP2, say. Easy, but a blunt instrument: if there are points elsewhere in your build where there is a lot of project parallelism but not much CL parallelism, or vice versa, you probably just slowed them down.<\/p>\n<h5>Tune \/MP for each project and configuration<\/h5>\n<p>A project that compiles at a relatively parallelized point in the build is not such a good candidate for \/MP, for example. You might adjust by configuration as well. Retail configuration can be much slower to build because the compiler\u2019s optimizing more: that might make it interesting to enable \/MP for Retail and not Debug.<\/p>\n<h5>Get super custom<\/h5>\n<p>In your team, you might have a range of hardware. Perhaps your developers have 2-CPU machines, but your nightly build is on an 8-CPU beast. Yet the both need to build the same set of sources, and you don&#8217;t want any box to be either slow or thrashing. In this case, you could use environment variables, and Conditions on the MSBuild tags. Almost all MSBuild tags can have Conditions. <\/p>\n<p>Here\u2019s an example below. When a property \u201cMultiprocCLCount\u201d (which I just invented) has a value, and it\u2019s greater than 0, \/MP is enabled with that value.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/4\/2019\/06\/image_42.png\"><img decoding=\"async\" title=\"image\" border=\"0\" alt=\"image\" src=\"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-content\/uploads\/sites\/4\/2010\/03\/image_thumb_20.png\" width=\"859\" height=\"93\"><\/a> <\/p>\n<p>MSBuild pulls in all environment variables as its initial properties when it starts up. So on my fast machine, I set an environment variable <strong>MultiprocCLCount=8<\/strong>, and on my developer boxes, I set <strong>MultiprocCLCount=2.<\/strong> <\/p>\n<p>The build machine\u2019s script could also parameterize the \/m switch going to MSBuild.exe, like<strong> \/m:%MultiprocMSBuildCount%<\/strong> <\/p>\n<p>To other properties that might be useful in exotic conditions: <strong>$(Number_Of_Processors) <\/strong>is the number of logical cores on the box \u2013 this just comes from the environment variable. <strong>$(MSBuildNodeCount) <\/strong>is the value that was passed to \/m on msbuild.exe, or within VS, the value from Tools&gt;Options for project parallelism.<\/p>\n<p>That\u2019s it. I hope while walking you through \/m and \/MP I\u2019ve also given you an overview of some MSBuild features and how much flexibility they give you to configure your build process. <\/p>\n<p>Optimizing your build speed is a huge topic so look for more blogging on this subject from me.<\/p>\n<p>Dan \u2013 Developer Lead, MSBuild<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A great way to get fast builds on a multiprocessor computer is to take advantage of as much parallelism in your build as possible. If you have C++ projects, there\u2019s two different kinds of parallelism you can configure. What are the dials I can set? Project-level parallel build, which is controlled by MSBuild, is set [&hellip;]<\/p>\n","protected":false},"author":13,"featured_media":255385,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[155],"tags":[1383,37,185,53],"class_list":["post-2363","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-visual-studio","tag-c","tag-msbuild","tag-node-js","tag-performance"],"acf":[],"blog_post_summary":"<p>A great way to get fast builds on a multiprocessor computer is to take advantage of as much parallelism in your build as possible. If you have C++ projects, there\u2019s two different kinds of parallelism you can configure. What are the dials I can set? Project-level parallel build, which is controlled by MSBuild, is set [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-json\/wp\/v2\/posts\/2363","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-json\/wp\/v2\/comments?post=2363"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-json\/wp\/v2\/posts\/2363\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-json\/wp\/v2\/media\/255385"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-json\/wp\/v2\/media?parent=2363"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-json\/wp\/v2\/categories?post=2363"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/visualstudio\/wp-json\/wp\/v2\/tags?post=2363"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}