The Coalition Sees 27.9X Iteration Build Improvement with Visual Studio 2019
Visual Studio 2019 v16.8 Preview 3.2 introduces significant build and link time improvements. In this blog post, we detail how the team in The Coalition building Gears 5 tested the compile and link times in three different versions of Visual Studio. The experiment was conducted to measure not only overall end-to-end time improvements but also link time improvements in the preview compared to the current release. Using Visual Studio 2019, the team saw 2.67x faster end-to-end build times and 27.9x faster link times compared to Visual Studio 2017.
“The improvements are massive and will greatly improve our iteration times” –Aurel Cordonnier from The Coalition
Compile times are a consistent theme of feedback from Visual Studio developers. Recently, we launched numerous build optimization efforts. One of which is working closely with our colleagues across Xbox Game Studios. Here, we seek to identify opportunities to improve our Visual Studio developer experience. The following details the build and link time improvements The Coalition saw when using Visual Studio 2019 v16.8 Preview 3.2.
Building Gears 5
The Gears 5 team compiled and linked the PC version of the game, using three different build configurations sequentially. The configs compile and link a very large code base into one monolithic executable. To produce consistent results, the team chose not to use /debug:fastlink with the build machines since the machines need to publish symbols.
To measure link times, the wall clock execution time of this compilation experiment starts before link is called and ends after linking the final executable. Similarly, the end-to-end times were read from the build system output (total execution time). Then, the link time from linker’s verbose output were subtracted from the total execution time to get the compile times. The results are an average of five tests conducted for each configuration to achieve consistency with a low variance.
The results of the experiment showed both compile and link times significantly improved in Visual Studio 2019. The compilation time for all three configurations has been improved by a factor of 2.15. In the ship configuration, Gears 5 took 41 minutes (2461.92 seconds) to compile in Visual Studio 2017. In comparison, the compilation took only 18.76 minutes (1125.61 seconds) in Visual Studio 2019 16.8 Preview 3.2. On average, the link times improved 27.9X. The ship config took only 29.57 seconds compared to 11.28 minutes (677.07 seconds) previously. Overall, the end-to-end compilation time decreased by a factor of 2.67X.
|Compile Time (seconds)||dev||test||ship|
|x times faster||2.08||2.18||2.19|
|average x times faster||2.15|
|Link Time (seconds)||dev||test||ship|
|x times faster||28.14||25.98||29.57|
|average x times faster||27.90|
|x times faster||2.59||2.70||2.73|
|average x times faster||2.67|
The C++ team made large steps towards Linker performance in Visual Studio version 16.0 and 16.2 with two significant changes to the linker that help speed up the generation of Program Database (PDB) files. Type pruning in the backend removes unreferenced type information. By using a fast hash function to identify identical types, type merging sped up. Please see details in this blog post.
In version 16.6, changes were made to algorithms inside the PDB and Debug Interface Access components. Next, optimization efforts were made in version 16.7 to speed up the worst case Incremental Linking time. Later, multi-threading the generation of PDB files further improved performance in version 16.8. Please stay tuned for future blogs that dive deeper into these improvements.
Aurel from The Coalition spoke to us about how improved build times have affected the studio.
What prompted the move to Visual Studio 2019?
So, we’re not actually moving Gears 5 to 2019. We did the work to get Visual Studio 2019 going on PC for the purpose of comparing build times and providing feedback. Gears 5 shipped over a year ago and the game has been very carefully tuned; therefore, we don’t want to take the risk of changing the toolchain as we have limited testing capacity and very slight optimization changes could change the balance of the game in very subtle ways. In addition, the XDK does not support 2019 so we would need to support two toolchains which is something we don’t have the capacity for.
The results of the experiment reinforced our desire to stay current with new versions of Visual Studio for our future projects.
What are some immediate benefits?
We are using Visual Studio 2019 on our next (unannounced) project and the benefits are many: faster compile/link times, more responsive UI, much more stable debugging.
What are some potential long-term benefits?
Developer iteration is key to our success, any time we can shave a few seconds off our iteration loop, it means developers have more time to write quality code, fix bugs etc. Faster build times on the build farm means our testing team can get a build sooner, can find more bugs etc.
In the end, we will end up with a better game for our fans.
How might this change the workflow patterns of the studio?
With Visual Studio 2017 (and 2019 prior to 16.8), we had to choose between fast link times or debugging stability (by using /debug:fastlink or not).
With 16.8, we can have both as link times without fastlink are very fast.
Upgrade Visual Studio
Download the latest Visual Studio 2019 Preview and experience faster end-to-end compile times for your own projects.