{"id":25996,"date":"2020-05-08T14:27:05","date_gmt":"2020-05-08T14:27:05","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/cppblog\/?p=25996"},"modified":"2020-05-25T02:15:30","modified_gmt":"2020-05-25T02:15:30","slug":"faster-builds-with-pch-suggestions-from-c-build-insights","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cppblog\/faster-builds-with-pch-suggestions-from-c-build-insights\/","title":{"rendered":"Faster builds with PCH suggestions from C++ Build Insights"},"content":{"rendered":"

The creation of a precompiled header (PCH)<\/a> is a proven strategy for improving build times. A PCH eliminates the need to repeatedly parse a frequently included header by processing it only once at the beginning of a build. The selection of headers to precompile has traditionally been viewed as a guessing game, but not anymore! In this article, we will show you how to use the vcperf analysis tool<\/a> and the C++ Build Insights SDK<\/a> to pinpoint the headers you should precompile for your project. We\u2019ll walk you through building a PCH for the open source Irrlicht<\/a> project, yielding a 40% build time improvement.<\/p>\n

How to obtain and use vcperf<\/strong><\/span><\/p>\n

The examples in this article make use of vcperf<\/em>, a tool that allows you to capture a trace of your build and to view it in the Windows Performance Analyzer (WPA). The latest version is available in Visual Studio 2019.<\/p>\n

1. Follow these steps to obtain and configure vcperf <\/em>and WPA:<\/strong><\/span><\/p>\n

    \n
  1. Download and install the latest Visual Studio 2019<\/a>.<\/li>\n
  2. Obtain WPA by downloading and installing the latest Windows ADK<\/a>.<\/li>\n
  3. Copy the perf_msvcbuildinsights.dll<\/em> file from your Visual Studio 2019\u2019s MSVC installation directory to your newly installed WPA directory. This file is the C++ Build Insights WPA add-in, which must be available to WPA for correctly displaying the C++ Build Insights events.\n
      \n
    1. MSVC\u2019s installation directory is typically: C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\{<\/span>Edition<\/span>}<\/span>\\VC\\Tools\\MSVC\\{Version}\\bin\\Hostx64\\x64<\/code>.<\/li>\n
    2. WPA\u2019s installation directory is typically: C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit<\/code>.<\/li>\n<\/ol>\n<\/li>\n
    3. Open the perfcore.ini<\/em> file in your WPA installation directory and add an entry for the perf_msvcbuildinsights.dll<\/em> file. This tells WPA to load the C++ Build Insights add-in on startup.<\/li>\n<\/ol>\n

      You can also obtain the latest vcperf<\/em> and WPA add-in by cloning and building the vcperf GitHub repository<\/a>. Feel free to use your built copy in conjunction with Visual Studio 2019!<\/p>\n

      2. Follow these steps to collect a trace of your build:<\/strong><\/span><\/p>\n

        \n
      1. Open an elevated x64 Native Tools Command Prompt for VS 2019<\/em><\/strong>.<\/li>\n
      2. Obtain a trace of your build:\n
          \n
        1. Run the following command: vcperf \/start MySessionName<\/code>.<\/li>\n
        2. Build your C++ project from anywhere, even from within Visual Studio (vcperf<\/em> collects events system-wide).<\/li>\n
        3. Run the following command: vcperf \/stop MySessionName outputFile.etl<\/code>. This command will stop the trace, analyze all events, and save everything in the outputFile.etl<\/em> trace file.<\/li>\n<\/ol>\n<\/li>\n
        4. Open the trace you just collected in WPA.<\/li>\n<\/ol>\n

          Viewing header parsing information in WPA<\/strong><\/span><\/p>\n

          C++ Build Insights provides a WPA view called Files<\/em><\/strong> that allows you to see the aggregated parsing time of all headers in your program. After opening your trace in WPA, you can open this view by dragging it from the Graph Explorer<\/em><\/strong> pane to the Analysis<\/em><\/strong> window, as shown below.<\/p>\n

          \"dragging<\/p>\n

          The most important columns in this view are the ones named Inclusive Duration <\/em><\/strong>and Count<\/em><\/strong>, which show the total aggregated parsing time of the corresponding header and the number of times it was included, respectively.<\/p>\n

          Case study: using vcperf and WPA to create a PCH for the Irrlicht 3D engine<\/strong><\/span><\/p>\n

          In this case study, we show how to use vcperf<\/em> and WPA to create a PCH for the Irrlicht open source project, making it build 40% faster.<\/p>\n

          Use these steps if you would like to follow along:<\/p>\n

            \n
          1. Clone the Irrlicht repository<\/a> from GitHub.<\/li>\n
          2. Checkout the following commit: 97472da9c22ae4a<\/code>.<\/li>\n
          3. Open an elevated x64 Native Tools Command Prompt for VS 2019 Preview <\/em><\/strong>command prompt and go to the location where you cloned the Irrlicht project.<\/li>\n
          4. Type the following command: devenv \/upgrade .\\source\\Irrlicht\\Irrlicht15.0.sln<\/code>. This will update the solution to use the latest MSVC.<\/li>\n
          5. Download and install the DirectX Software Development Kit<\/a>. This SDK is required to build the Irrlicht project.\n
              \n
            1. To avoid an error, you may need to uninstall the Microsoft Visual C++ 2010 x86 Redistributable and Microsoft Visual C++ 2010 x64 Redistributable components from your computer before installing the DirectX SDK. You can do so from the Add and remove programs<\/em><\/strong> settings page in Windows 10. They will be reinstalled by the DirectX SDK installer.<\/li>\n<\/ol>\n<\/li>\n
            2. Obtain a trace for a full rebuild of Irrlicht. From the repository\u2019s root, run the following commands:\n
                \n
              1. vcperf \/start Irrlicht<\/code>. This command will start the collection of a trace.<\/li>\n
              2. msbuild \/m \/p:Platform=x64 \/p:Configuration=Release .\\source\\Irrlicht\\Irrlicht15.0.sln \/t:Rebuild \/p:BuildInParallel=true<\/code>. This command will rebuild the Irrlicht project.<\/li>\n
              3. vcperf \/stop Irrlicht irrlicht.etl<\/code>. This command will save a trace of the build in irrlicht.etl<\/em>.<\/li>\n<\/ol>\n<\/li>\n
              4. Open the trace in WPA.<\/li>\n<\/ol>\n

                We open the Build Explorer<\/em><\/strong> and Files<\/em><\/strong> views one on top of the other, as shown below. The Build Explorer<\/em><\/strong> view indicates that the build lasted around 57 seconds. This can be seen by looking at the time axis at the bottom of the view (labeled A). The Files<\/em><\/strong> view shows that the headers with the highest aggregated parsing time were Windows.h<\/em> and irrAllocator.h<\/em> (labeled B). They were parsed 45 and 217 times, respectively.<\/p>\n

                \"Files<\/p>\n

                We can see where these headers were included from by rearranging the columns of the Files<\/em><\/strong> view to group by the IncludedBy<\/em><\/strong> field. This action is shown below.<\/p>\n

                \"Using<\/p>\n

                Creating a PCH<\/strong><\/span><\/p>\n

                We first add a new pch.h<\/em> file at the root of the solution. This header contains the files we want to precompile, and will be included by all C and C++ files in the Irrlicht solution. We only add the irrAllocator.h<\/em> header when compiling C++ because it\u2019s not compatible with C.<\/p>\n

                \"Precompiled<\/p>\n

                PCH files must be compiled before they can be used. Because the Irrlicht solution contains both C and C++ files, we need to create 2 versions of the PCH. We do so by adding the pch-cpp.cpp<\/em> and pch-c.c<\/em> files at the root of the solution. These files contain nothing more than an include directive for the pch.h<\/em> header we created in the previous step.<\/p>\n

                \"Precompiled<\/p>\n

                We modify the Precompiled Headers<\/em><\/strong> properties of the pch-cpp.cpp<\/em> and pch-c.c<\/em> files as shown below. This will tell Visual Studio to create our 2 PCH files.<\/p>\n

                \"Changing<\/p>\n

                We modify the Precompiled Headers<\/em><\/strong> properties for the Irrlicht project as shown below. This will tell Visual Studio to use our C++ PCH when compiling the solution.<\/p>\n

                \"Using<\/p>\n

                We modify the Precompiled Headers<\/em><\/strong> properties for all C files in the solution as follows. This tells Visual Studio to use the C version of the PCH when compiling these files.<\/p>\n

                \"Using<\/p>\n

                In order for our PCH to be used, we need to include the pch.h header in all our C and C++ files. For simplicity, we do this by modifying the Advanced<\/em> C\/C++ properties<\/em><\/strong> for the Irrlicht project to use the \/FI<\/code> compiler option. This change results in pch.h<\/em> being included at the beginning of every file in the solution even if we don\u2019t explicitly add an include directive.<\/p>\n

                \"pch.h<\/p>\n

                A couple of code fixes need to be applied for the project to build correctly following the creation of our PCH:<\/p>\n

                  \n
                1. Add a preprocessor definition for HAVE_BOOLEAN for the entire Irrlicht project.<\/li>\n
                2. Undefine the far preprocessor definition in 2 files.<\/li>\n<\/ol>\n

                  For the full list of changes, see our fork on GitHub<\/a>.<\/p>\n

                  Evaluating the final result<\/strong><\/span><\/p>\n

                  After creating the PCH, we collect a new vcperf<\/em> trace of a full rebuild of Irrlicht by following the steps in the Case study: using vcperf and WPA to create a PCH for an open source project <\/em><\/strong>section. <\/em><\/strong>We notice that the build time has gone from 57 seconds to 35 seconds, an improvement of around 40%. We also notice that Windows.h<\/em> and irrAllocator.h<\/em> no longer show up in the Files<\/em><\/strong> view as top contributors to parsing time.<\/p>\n

                  \"Windows.h<\/p>\n

                  Getting PCH suggestions using the C++ Build Insights SDK<\/strong><\/span><\/p>\n

                  Most analysis tasks performed manually with vcperf<\/em> and WPA can also be performed programmatically using the C++ Build Insights SDK. As a companion to this article, we\u2019ve prepared the TopHeaders<\/em><\/strong> SDK sample. It prints out the header files that have the highest aggregated parsing times, along with their percentage weight in relation to total compiler front-end time. It also prints out the total number of translation units each header is included in.<\/p>\n

                  Let\u2019s repeat the Irrlicht case study from the previous section, but this time by using the TopHeaders <\/em><\/strong>sample to see what it finds. <\/em>Use these steps if you want to follow along:<\/p>\n

                    \n
                  1. Clone the C++ Build Insights SDK samples GitHub repository<\/a> on your machine.<\/li>\n
                  2. Build the Samples.sln <\/em>solution, targeting the desired architecture (x86 or x64), and using the desired configuration (debug or release). The sample\u2019s executable will be placed in the out\/{architecture}\/{configuration}\/TopHeaders<\/code> folder, starting from the root of the repository.<\/li>\n
                  3. Follow the steps from the Case study: using vcperf and WPA to create a PCH for the Irrlicht 3D engine<\/em><\/strong> section to collect a trace of the Irrlicht solution rebuild. Use the vcperf \/stopnoanalyze Irrlicht irrlicht-raw.etl<\/code> command instead of the \/stop<\/code> command when stopping your trace. This will produce an unprocessed trace file that is suitable to be used by the SDK.<\/li>\n
                  4. Pass the irrlicht-raw.etl<\/em> trace as the first argument to the TopHeaders<\/em><\/strong> executable.<\/li>\n<\/ol>\n

                    As shown below, TopHeaders<\/em><\/strong> correctly identifies both Windows.h<\/em> and irrAllocator.h<\/em> as top contributors to parsing time. We can see that they were included in 45 and 217 translation units, respectively, as we had already seen in WPA.<\/p>\n

                    <\/p>\n

                    Rerunning TopHeaders <\/em><\/strong>on our fixed codebase shows that the Windows.h<\/em> and irrAllocator.h<\/em> headers are no longer a concern. We see that several other headers have also disappeared from the list. These headers are referenced by irrAllocator.h<\/em>, and were included in the PCH by proxy of irrAllocator.h<\/em>.<\/p>\n

                    <\/p>\n

                    Understanding the sample code<\/strong><\/span><\/p>\n

                    We first filter all stop activity events and only keep front-end file and front-end pass events. We ask the C++ Build Insights SDK to unwind the event stack for us in the case of front-end file events. This is done by calling MatchEventStackInMemberFunction<\/code>, which will grab the events from the stack that match the signature of TopHeaders::OnStopFile<\/code>. When we have a front-end pass event, we simply keep track of total front-end time directly.<\/p>\n

                    AnalysisControl OnStopActivity(const EventStack& eventStack) override\r\n{\r\n    switch (eventStack.Back().EventId())\r\n    {\r\n    case EVENT_ID_FRONT_END_FILE:\r\n        MatchEventStackInMemberFunction(eventStack, this, \r\n            &TopHeaders::OnStopFile);\r\n        break;\r\n\r\n    case EVENT_ID_FRONT_END_PASS:\r\n        \/\/ Keep track of the overall front-end aggregated duration.\r\n        \/\/ We use this value when determining how significant is\r\n        \/\/ a header's total parsing time when compared to the total\r\n        \/\/ front-end time.\r\n        frontEndAggregatedDuration_ += eventStack.Back().Duration();\r\n        break;\r\n\r\n    default:\r\n        break;\r\n    }\r\n\r\n    return AnalysisControl::CONTINUE;\r\n}<\/pre>\n
                    We use the OnStopFile<\/code> function to aggregate parsing time for all headers into our std::unordered_map fileInfo_<\/code> structure. We also keep track of the total number of translation units that include the file, as well as the path of the header.<\/p>\n
                    AnalysisControl OnStopFile(FrontEndPass fe, FrontEndFile file)\r\n{\r\n    \/\/ Make the path lowercase for comparing\r\n    std::string path = file.Path();\r\n\r\n    std::transform(path.begin(), path.end(), path.begin(),\r\n        [](unsigned char c) { return std::tolower(c); });\r\n\r\n    auto result = fileInfo_.try_emplace(std::move(path), FileInfo{});\r\n\r\n    auto it = result.first;\r\n    bool wasInserted = result.second;\r\n\r\n    FileInfo& fi = it->second;\r\n\r\n    fi.PassIds.insert(fe.EventInstanceId());\r\n    fi.TotalParsingTime += file.Duration();\r\n\r\n    if (result.second) {\r\n        fi.Path = file.Path();\r\n    }\r\n\r\n    return AnalysisControl::CONTINUE;\r\n}<\/pre>\n
                    At the end of the analysis, we print out the information that we have collected for the headers that have the highest aggregated parsing time.<\/p>\n
                    AnalysisControl OnEndAnalysis() override\r\n{\r\n    using namespace std::chrono;\r\n\r\n    auto topHeaders = GetTopHeaders();\r\n\r\n    if (headerCountToDump_ == 1) {\r\n        std::cout << \"Top header file:\";\r\n    }\r\n    else {\r\n        std::cout << \"Top \" << headerCountToDump_ <<\r\n            \" header files:\";\r\n    }\r\n\r\n    std::cout << std::endl << std::endl;\r\n\r\n    for (auto& info : topHeaders)\r\n    {\r\n        double frontEndPercentage = \r\n            static_cast<double>(info.TotalParsingTime.count()) \/\r\n            frontEndAggregatedDuration_.count() * 100.;\r\n\r\n        std::cout << \"Aggregated Parsing Duration: \" <<\r\n            duration_cast<milliseconds>(\r\n                info.TotalParsingTime).count() << \r\n            \" ms\" << std::endl;\r\n        std::cout << \"Front-End Time Percentage:   \" <<\r\n            std::setprecision(2) << frontEndPercentage << \"% \" << \r\n            std::endl;\r\n        std::cout << \"Inclusion Count:             \" <<\r\n            info.PassIds.size() << std::endl;\r\n        std::cout << \"Path: \" <<\r\n            info.Path << std::endl << std::endl;\r\n    }\r\n\r\n    return AnalysisControl::CONTINUE;\r\n}<\/pre>\n
                    Tell us what you think!<\/strong><\/span><\/p>\n
                    We hope the information in this article has helped you understand how to use C++ Build Insights to create new precompiled headers, or to optimize existing ones.<\/p>\n
                    Give vcperf<\/em> a try today by downloading the latest version of Visual Studio 2019<\/a>, or by cloning the tool directly from the vcperf Github repository<\/a>. Try out the TopHeaders<\/em><\/strong> sample from this article by cloning the C++ Build Insights samples repository<\/a> from GitHub, or refer to the official C++ Build Insights SDK documentation<\/a> to build your own analysis tools.<\/p>\n
                    Have you been able to improve your build times with the header file information provided by vcperf<\/em> or the C++ Build Insights SDK? Let us know in the comments below, on Twitter (@VisualC<\/a>), or via email at visualcpp@microsoft.com<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"
                    The creation of a precompiled header (PCH) is a proven strategy for improving build times. A PCH eliminates the need to repeatedly parse a frequently included header by processing it only once at the beginning of a build. The selection of headers to precompile has traditionally been viewed as a guessing game, but not anymore! […]<\/p>\n","protected":false},"author":6966,"featured_media":26021,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1,239],"tags":[],"class_list":["post-25996","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cplusplus","category-diagnostics"],"acf":[],"blog_post_summary":"
                    The creation of a precompiled header (PCH) is a proven strategy for improving build times. A PCH eliminates the need to repeatedly parse a frequently included header by processing it only once at the beginning of a build. The selection of headers to precompile has traditionally been viewed as a guessing game, but not anymore! […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts\/25996","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/users\/6966"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/comments?post=25996"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts\/25996\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/media\/26021"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/media?parent=25996"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/categories?post=25996"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/tags?post=25996"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}