Introducing C++ Build Insights

Avatar

Kevin

C++ builds should always be faster. In Visual Studio 2019 16.2, we’ve shown our commitment to this ideal by speeding up the linker significantly. Today, we are thrilled to announce a new collection of tools that will give you the power to make improvements of your own. If you’ve ever had time for breakfast while building C++, then you may have asked yourself: what is the compiler doing? C++ Build Insights is our latest take at answering this daunting question and many others. By combining new tools with tried-and-tested Event Tracing for Windows (ETW), we’re making timing information for our C++ toolchain more accessible than ever before. Use it to make decisions tailored to your build scenarios and improve your build times.

Getting started

C++ Build Insights makes use of vcperf, a tool that allows you to capture a trace of your build and to view it in the Windows Performance Analyzer (WPA). The latest version of vcperf is available in Visual Studio 2019, but WPA must be obtained separately.

1. Follow these steps to obtain and configure vcperf and WPA:

  1. Download and install the latest Visual Studio 2019.
  2. Obtain WPA by downloading and installing the latest Windows ADK.
  3. Copy the perf_msvcbuildinsights.dll file from your Visual Studio 2019’s MSVC installation directory to your newly installed WPA directory. This file is the C++ Build Insights WPA add-in, which must be available to WPA for correctly displaying the C++ Build Insights events.
    1. MSVC’s installation directory is typically: C:\Program Files (x86)\Microsoft Visual Studio\2019\{Edition}\VC\Tools\MSVC\{Version}\bin\Hostx64\x64.
    2. WPA’s installation directory is typically: C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit.
  4. Open the perfcore.ini file in your WPA installation directory and add an entry for the perf_msvcbuildinsights.dll file. This tells WPA to load the C++ Build Insights add-in on startup.

You can also obtain the latest vcperf and WPA add-in by cloning and building the vcperf GitHub repository. Feel free to use your built copy in conjunction with Visual Studio 2019!

2. Follow these steps to collect a trace of your build:

  1. Open an elevated x64 Native Tools Command Prompt for VS 2019.
  2. Obtain a trace of your build:
    1. Run the following command: vcperf /start MySessionName.
    2. Build your C++ project from anywhere, even from within Visual Studio (vcperf collects events system-wide).
    3. Run the following command: vcperf /stop MySessionName outputFile.etl. This command will stop the trace, analyze all events, and save everything in the outputFile.etl trace file.
  3. Open the trace you just collected in WPA.

At the crux of the matter with the Build Explorer

Central to understanding a build is having an overview of what happened through time. The Build Explorer is a core element of C++ Build Insights that we designed for this purpose. Drag it from the Graph Explorer panel on the left onto the main analysis window in WPA. Use it to diagnose parallelism issues, detect bottlenecks, and determine if your build is dominated by parsing, code generation, or linking.

 

Gif showing the tool in action

 

A thousand scoffs for the thousand cuts: aggregated statistics

The most insidious of build time problems are the ones that strike you little by little until the damage is so large that you wonder what hit you. A common example is the repetitive parsing of the same header file due to its inclusion in many translation units. Each inclusion may not have a large impact by itself but may have devastating effects on build times when combined with others. C++ Build Insights provides aggregated statistics to assist you in warding off such threats. An example of this capability is illustrated below, where file parsing statistics are aggregated over an entire build to determine the most time-consuming headers. To view this information for your build, choose the Files view from the Graph Explorer panel on the left in WPA. Drag it to the analysis window.

 

Analysis window showing the number of times a header was included and the accumulative time it took.

 

What information can you get?

With C++ Build Insights, expect to be able to obtain the following information for all your compiler and linker invocations.

For the front-end: Overall front-end time, individual files parsing time. For the back-end: overall back-end time, function optimization time, whole program analysis time, code generation thread time, whole program analysis thread time. For the general compiler: overall compiler time, command line, inputs/outputs, working directory, tool path. For the general linker: overall linker time, pass 1 time, pass 2 time, link-time code generation time, command line, inputs/outputs, working directory, tool path.

Tell us what you think!

Download your copy of the latest Visual Studio 2019, and get started with C++ Build Insights today! For a complete set of documentation, see the Get started with C++ Build Insights page.

In this article, we shared two features with you, but there is much more to come! Stay tuned for more blog posts detailing specific ways in which you can use C++ Build Insights to improve your builds.

We look forward to hearing from you on how you used vcperf and WPA to understand and optimize your builds. Are there any other pieces of information you would like to be able to get from C++ Build Insights? Tell us what you think in the comments below. You can also get in touch with us at visualcpp@microsoft.com, or on Twitter @VisualC.

33 comments

Comments are closed. Login to edit/delete your existing comments

  • Avatar
    Alf P. Steinbach

    What’s needed for faster builds is a way to compile any number of translation units with the *assumption* of a fixed set of preprocessor definitions, because

    then a header (with the exception of designated ones such as <assert.h>) needs only be compiled in the first translation unit it's encountered.

    There is already machinery for this in Visual C++'s support for a single precompiled header.

    When I've mentioned this before the answer's always been, let's wait for modules.

    That's not an adult level answer.

    • Avatar
      Tony Riviere

      This should be possible with C++20 Modules where you can transform a #include directive into a import directive. By doing this, the header will be compiled once (per module?) instead of being copy/paste into the source TU.

  • Avatar
    Peter Gerell

    Hi. This looks interesting. However I have problems capturing information for my project.
    When I stop my tracing session I get the following error:
    “Failed to stop trace. Events were dropped during the trace. Please try recollecting the trace.”
    I believe this happens when one project that has custom build steps to download and build boost is being built.
    Is there some public issue tracker where I can report this?

    • Avatar
      Kevin CadieuxMicrosoft employee

      Hi Peter,

      You can report issues on Developer Community. If you do end up opening an issue, submitting a raw trace would help us with the diagnosis. A raw trace can be captured by using /stopnoanalyze instead of /stop when stopping the trace.

      Is there an easy way you can disable the custom step that builds Boost to see if anything changes? If not, you could try the following:

      1. Add the /nocpusampling switch to your vcperf start command: vcperf /start /nocpusampling MySession
      2. If that still doesn’t work, add the /basic switch to your start command: vcperf /start /nocpusampling /basic MySession

      Both of these options will result in less events being collected and will reduce the chances of events being dropped. However, they will also result in less information being available when viewing the trace in WPA. Option 2 will disable the Files and Functions views, leaving only the Build Explorer.

  • Avatar
    rob crompton

    “…will work with any build system…”: I was going to ask whether this means it will work with older versions of Visual Studio. We are using VS2017 and aren’t likely to upgrade soon.
    So I tried it and it does work 🙂 Great!

    • Avatar
      Toni M

      How can it work if the cl ETW event emitters aren’t there? Or have they existed all along? Do you get a reduced trace only showing the process invocations?

      • Avatar
        Kevin CadieuxMicrosoft employee

        Hi Toni,

        Rob has discovered something that is not yet announced: vcperf is backward compatible down to VS 2017. The reason is that VS 2017 contains most of the ETW events required to get a reasonable trace. I will touch more on that in a later blog post, as well as make several other announcements.

    • Avatar
      Kevin CadieuxMicrosoft employee

      Hi Vaclav,

      When this happens it’s typically due to one of the following reasons:

      1. The wrong version of WPA is used. Make sure you download the one from the ADK preview which contains the C++ Build Insights plugin. You can download it here: https://www.microsoft.com/en-us/software-download/windowsinsiderpreviewADK
      2. The C++ Build Insights views are disabled in WPA. To enable them, select Windows –> Select Tables, and check all C++ Build Insights views under the Diagnostics category. Restart WPA.
      3. The version of the MSVC toolset being traced is too old and does not support C++ Build Insights. Make sure you are using the C++ toolchain that comes with VS 2019.

      If you have tried all of the above and it is still not working, please feel free to open a bug on Developer Community at https://developercommunity.visualstudio.com/

  • Avatar
    Tim Wynants

    Looks very useful and would like to try it. unfortunately, I can’t start the tracing.

    Microsoft (R) Visual C++ (R) Performance Analyzer 1.2.19101801
    Starting tracing session MySession…
    Failed to start trace.
    A trace that is currently being collected on your system is preventing vcperf from starting a new one. This can occur if you forgot to stop a vcperf trace prior to running the start command, or if processes other than vcperf have started ETW traces of their own. Please try running the vcperf /stop or /stopnoanalyze commands on your previously started traces. …

    – Never started a trace
    – Restarted the OS many times
    – Tried to stop all entries off ‘tracelog -l’ but none of those could be stopped by ‘vcperf /stop’

    Nothing helps …

    • Avatar
      Kevin CadieuxMicrosoft employee

      Hi Tim,

      Collection can also be prevented if something on your system is currently collecting a kernel trace. A kernel trace would show up in your ‘tracelog -l’ as NT Kernel Logger. You can stop this one by running the ‘xperf -stop’ command, though I would first try to understand why a kernel trace is being collected to begin with.

      Please let me know if you need more help with this.

      Thanks,

      Kevin

  • ITISAGLN
    ITISAGLN

    Hello, World.
    I don’t see Build Explorer. Under Graph Explorer, I see System Activity/Images, Generic Events, Regions, Threads, and Computation/CPU/DPC/Flame/etc..
    I am not using “Preview”, I am using Professional, so I had to splice together various blog posts.
    I have this file C:\Users\..me..\Documents\Visual Studio 2019\..my proj..\Projects\packages\Microsoft.Cpp.BuildInsights.1.0.0\wpa\x64\perf_msvcbuildinsights.dll
    I added perf_msvcbuildinsights.dll to C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\perfcore.ini
    I vcperf /start MySessionName
    Then rebuilt my proj…
    Then I vcperf /stopnoanalyze MySessionName outputTraceFile.etl (because https://docs.microsoft.com/en-us/cpp/build-insights/reference/sdk/overview?view=vs-2019: “Important Use the /stopnoanalyze command when stopping your trace with vcperf. You can’t use the C++ Build Insights SDK to analyze traces stopped by the regular /stop command.”)
    Any idea why I don’t see Build Explorer?

    • Avatar
      Kevin CadieuxMicrosoft employee

      Hello ITISAGLN,

      Sorry for the confusion about the /stopnoanalyze command. This command is only used to analyze traces programmatically via the SDK. You should use the regular /stop command if you are going to view a trace in WPA, even if the perf_msvcbuildinsights.dll file comes from the NuGet package.

      I hope this helps! Please let me know if you are still having problems.

      • ITISAGLN
        ITISAGLN

        Thanks, Kevin. I was originally using /stop. Build Explorer didn’t show so I went looking for answers and apparently landed on wrong one (/stopnoanalyze command). I just repeated your steps 1-3 and there is no Build Explorer; Only Graph Explorer (subcategories are System Activity and Configuration). Any idea on what I should try next to see Build Explorer?

        • Avatar
          Kevin CadieuxMicrosoft employee

          This could be caused by the following:

          1. Using an unsupported toolset. Events are only present in Visual Studio 2017 version 15.7 and above. The build that you are tracing must be using at least this toolset.
          2. The WPA add-in was not installed correctly. Try opening WPA and click on Window -> Select Tables. Search for the Diagnostics category and you should see 4 C++ Build Insights views listed there. Do you see that? If not, you might want to try reinstalling the WPA add-in.

          Based on your description I think your problem might be #1. Could you please confirm your build is using at least VS 2017 15.7? If you are still having issues, please open a bug on Developer Community so that I can help you further: https://developercommunity.visualstudio.com/content/problem/post.html?space=62

          Thanks!

          • ITISAGLN
            ITISAGLN

            I am on Version 16.5.5.
            My WPA does not have Window/Tables; only New Analysis View, New Comp. Analysis View, Duplicate View, Rename View, Close View, Close All Views, Diagnostic Console, Details, Analysis Assistant, My Presets, Go Back to Analysis.
            I see and clicked “Diagnostic Console”:
            Error: Could not load add-in: perf_msvcbuildinsights.dll. Exception: ..
            WUT?
            It’s here: I copied C:\Users\..me..\Documents\Visual Studio 2019\..my proj..\Projects\packages\Microsoft.Cpp.BuildInsights.1.0.0\wpa\x64\perf_msvcbuildinsights.dll -> C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit

            I’ll try reinstalling WPA…

            Thanks!

          • ITISAGLN
            ITISAGLN

            This tool looks like exactly what C++-ers need. Nice job. My first discovery for my project is a sad one and reveal architectural issues. The number one included file is a god-class. I knew it did too much but did not know it’d be #1 by as much as it is…I’m going to break it down. Thanks for this tool and I’ll keep using it!