{"id":27464,"date":"2021-02-02T17:00:17","date_gmt":"2021-02-02T17:00:17","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/cppblog\/?p=27464"},"modified":"2023-02-02T19:08:53","modified_gmt":"2023-02-02T19:08:53","slug":"playground-games-and-turn-10-studios-see-18-2x-and-4-95x-link-time-improvements-respectively-on-visual-studio-2019","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cppblog\/playground-games-and-turn-10-studios-see-18-2x-and-4-95x-link-time-improvements-respectively-on-visual-studio-2019\/","title":{"rendered":"Playground Games and Turn 10 Studios See 18.2X and 4.95X Link Time Improvements Respectively on Visual Studio 2019"},"content":{"rendered":"

Introduction<\/h2>\n

\"Image<\/p>\n

The C++ team at Visual Studio has delivered substantial build and link time improvements throughout Visual Studio 2019. This blog is Part 2 of a series of blogs showcasing real-world results of our efforts. See how the Gears 5 team benefited from iteration build time improvements in Part 1.<\/a><\/p>\n

In this blog, the Forza Horizon 4 team shares tests results of compile and link times in three different versions of Visual Studio. Link times are now 18.15X faster<\/strong> than in Visual Studio 2017. Likewise, Turn 10 Studios saw a 4.95X<\/strong> improvement in link time. The decrease in build time enabled Playground Games to switch from \/debug:fastlink<\/em> to\/debug:full<\/em>. Analyzing the results, we can see not only massive iteration build time wins but also applicable real-world quality of life improvements.<\/p>\n

\u201cUsing full linking removes the debugger stalls associated with fastlink and allows our engineering team to focus on debugging the current build rather than waiting for the development environment to become responsive. Coupled with the overall improvements to build and link times, this leads to less workflow interruption and productivity improvements across the team.\u201d<\/strong><\/em> \u2013 Andrew Sage from Playground Games<\/p>\n

<\/h2>\n

Building Forza Horizon 4<\/h2>\n

\"Image<\/a><\/p>\n

<\/h3>\n

Methodology<\/h3>\n

The Forza Horizon 4 team at Playground Games measured build times using Incredibuild\u2019s build monitor. The monitor shows wall clock time for each stage of the process. Compile time is measured at the start of the first file compile to immediately before linking. Link time is measured from start of the link step to the end of link step. Total time, measured by the start-to-end Incredibuild process, may be longer than compile and link times combined since some startup and post link steps are included in the measurement.<\/p>\n

Tests were conducted during quiet times at the studio to ensure parallel compiles were not affected by network load. To further reduce noise in data, tests were averaged over multiple runs, and were started after all hard drive disk activity had finished and Intellisense had finished parsing.<\/p>\n

<\/h3>\n

Results<\/h3>\n

Version 15.9.4 \/debug:fastlink<\/strong> vs 16.8 \/debug:full<\/strong><\/h4>\n

Comparing \/debug:fastlink<\/em> in Visual Studio 2017 and \/debug:full<\/em> in Visual Studio 2019 demonstrated real-world quality of life improvements. At first sight, total time has improved by a factor of 1.26<\/strong>; however, considerable time of 30-45 seconds<\/strong> per debug session was saved by switching to \/debug:full<\/em>, resulting in large productivity gains. The engineers at Playground Games have made the switch to \/debug:full<\/em> in order to both enjoy shorter build times and take advantage of the faster debugging experience afforded by \/debug:full<\/em>.<\/p>\n

\"Image<\/a><\/p>\n

 <\/p>\n\n\n\n\n\n\n\n
Total Time (minutes)<\/strong><\/td>\nFull rebuild<\/strong><\/td>\nSingle file change<\/strong><\/td>\n<\/tr>\n
VS2017 15.9.4 \/debug:fastlink<\/strong><\/td>\n4:22<\/td>\n1:39<\/td>\n<\/tr>\n
VS2019 16.8\u00a0\u00a0\u00a0 \/debug:full<\/strong><\/td>\n3:40<\/td>\n1:15<\/td>\n<\/tr>\n
x times faster <\/em><\/td>\n1.19<\/td>\n1.32<\/td>\n<\/tr>\n
average x times faster<\/em><\/strong><\/td>\n1.26<\/strong><\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

<\/h4>\n

Version 15.9.4 vs 16.8 using \/debug:full<\/h4>\n

In the second test, a comparison of builds using \/debug:full<\/em> was done over 3 versions of Visual Studio. The compilation time has been improved by a factor of 1.52. <\/strong>When doing a full rebuild, the compile time decreased from 4 minutes 39 seconds in Visual Studio 2017 to 3 minutes 5 seconds in Visual Studio 2019 16.8. For a single file change, a 19 second reduction was observed. On average, the link times improved by a factor of 18.15X. <\/strong>The full rebuild link time in Visual Studio 2019 16.8 decreased by 10 minutes 18 seconds<\/strong>, a monumental improvement.<\/p>\n

\"Image<\/a><\/p>\n

 <\/p>\n\n\n\n\n\n\n\n\n
Compile Time (minutes)<\/strong><\/td>\nFull rebuild<\/strong><\/td>\nSingle file change<\/strong><\/td>\n<\/tr>\n
VS2017 15.9.4 \/debug:full<\/td>\n4:39<\/td>\n0:55<\/td>\n<\/tr>\n
VS2019 16.7.2 \/debug:full<\/td>\n3:31<\/td>\n0:37<\/td>\n<\/tr>\n
VS2019 16.8 \u00a0\u00a0\u00a0\/debug:full<\/td>\n3:05<\/td>\n0:36<\/td>\n<\/tr>\n
x times faster<\/em><\/td>\n1.51<\/td>\n1.53<\/td>\n<\/tr>\n
average x times faster<\/em><\/strong><\/td>\n1.52<\/strong><\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

\"Image<\/a><\/p>\n

 <\/p>\n\n\n\n\n\n\n\n\n
Link Time (minutes)<\/strong><\/td>\nFull rebuild<\/strong><\/td>\nSingle file change<\/strong><\/td>\n<\/tr>\n
VS2017 15.9.4 \/debug:full<\/td>\n10:53<\/td>\n11:28<\/td>\n<\/tr>\n
VS2019 16.7.2 \/debug:full<\/td>\n1:23<\/td>\n1:39<\/td>\n<\/tr>\n
VS2019 16.8 \u00a0\u00a0\u00a0\/debug:full<\/td>\n0:35<\/td>\n0:39<\/td>\n<\/tr>\n
x times faster<\/em><\/td>\n18.66<\/strong><\/td>\n17.64<\/strong><\/td>\n<\/tr>\n
average x times faster<\/em><\/strong><\/td>\n18.15<\/strong><\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

\"Image<\/a><\/p>\n\n\n\n\n\n\n\n\n
Total Time (minutes)<\/strong><\/td>\nFull rebuild<\/strong><\/td>\nSingle file change<\/strong><\/td>\n<\/tr>\n
VS2017 15.9.4 \/debug:full<\/td>\n15:32<\/td>\n12:23<\/td>\n<\/tr>\n
VS2019 16.7.2 \/debug:full<\/td>\n4:54<\/td>\n3:40<\/td>\n<\/tr>\n
VS2019 16.8 \u00a0\u00a0\u00a0\/debug:full<\/td>\n3:40<\/td>\n1:15<\/td>\n<\/tr>\n
x times faster <\/em><\/td>\n4.24<\/td>\n9.91<\/td>\n<\/tr>\n
average x times faster<\/em><\/strong><\/td>\n7.07<\/strong><\/td>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Building Forza Motorsport<\/h2>\n

Methodology<\/h3>\n

At Turn 10 Studios, the Forza Motorsport team used MSBuild\u2019s Structured Log Viewer to obtain build and compile times. Each of the times below are the result of an average over 4 runs. CL refers to cl.exe (MSVC Compiler) and Lib refers to lib.exe (Microsoft Library Manager). While Full Build indicates total time, Debug, CL, and Lib times were chosen to be showcased; each time was averaged individually.<\/p>\n

Results<\/h3>\n

Version 15.9 vs 16.7 Profile Build with \/debug:fastlink<\/h4>\n

When doing a full Profile build with \/debug:fastlink, we can see a moderate improvement in build time while link time has improved by a factor of 1.96<\/strong>. Lib time has been decreased by 9.34X<\/strong>.<\/p>\n

\"Image<\/a><\/p>\n

 <\/p>\n\n\n\n\n\n\n
Time (seconds)<\/strong><\/td>\nFull Build (Profile)<\/strong><\/td>\nDebug Fastlink<\/strong><\/td>\nCL<\/strong><\/td>\nLib<\/strong><\/td>\n<\/tr>\n
VS2017 15.9<\/td>\n285.5<\/td>\n18<\/td>\n38.9<\/td>\n43.9<\/td>\n<\/tr>\n
VS2019 16.7<\/td>\n249.75<\/td>\n9.2<\/td>\n40.5<\/td>\n4.7<\/td>\n<\/tr>\n
x times faster <\/em><\/td>\n1.14<\/td>\n1.96<\/td>\n0.96<\/td>\n9.34<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Version 15.9 vs 16.7 Profile Build with \/debug:fulllink<\/h4>\n

Using \/debug:fulllink, we can see much more favorable numbers. Full build time is now 1.71X<\/strong> faster. Similarly, link time has been improved by a factor of 4.95<\/strong>. Lib time was also improved by 10.86X<\/strong>.<\/p>\n

\"Image<\/a><\/p>\n

 <\/p>\n\n\n\n\n\n\n
Time (seconds)<\/strong><\/td>\nFull Build (Profile)<\/strong><\/td>\nDebug Fulllink<\/strong><\/td>\nCL<\/strong><\/td>\nLib<\/strong><\/td>\n<\/tr>\n
VS2017 15.9<\/td>\n431<\/td>\n172<\/td>\n34<\/td>\n38<\/td>\n<\/tr>\n
VS2019 16.7<\/td>\n251.75<\/td>\n34.75<\/td>\n34.75<\/td>\n3.5<\/td>\n<\/tr>\n
x times faster <\/em><\/td>\n1.71<\/td>\n4.95<\/td>\n0.98<\/td>\n10.86<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

<\/h3>\n

Version 15.9 vs 16.7 Release Build<\/h4>\n

For the Release Build, we see similar improvements as above. Build time decreased by 1.81X<\/strong> and link time improved by a factor of 2.44<\/strong>.<\/p>\n

\"Image<\/a><\/p>\n

 <\/p>\n\n\n\n\n\n\n
Time (seconds)<\/strong><\/td>\nFull Build (Release)<\/strong><\/td>\nLink<\/strong><\/td>\nCL<\/strong><\/td>\nLib<\/strong><\/td>\n<\/tr>\n
VS2017 15.9<\/td>\n664<\/td>\n479<\/td>\n30<\/td>\n34.5<\/td>\n<\/tr>\n
VS2019 16.7<\/td>\n367<\/td>\n196<\/td>\n29<\/td>\n29<\/td>\n<\/tr>\n
x times faster <\/em><\/td>\n1.81<\/td>\n2.44<\/td>\n1.03<\/td>\n1.19<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

<\/h2>\n

Analysis<\/h2>\n

The significant build iteration time improvement is a direct result of optimization efforts by the C++ Team. Linker performance was improved in version 16.0 and 16.2 by speeding up Program Database (PDB) file generation and type merging<\/a>. Algorithmic changes in 16.6 and worse case Incremental Linking improvements in 16.7 increased optimization of Linker performance<\/a>. \u00a0In version 16.8, multi-threading generation of PDB files further improved performance. Stay tuned for a technical blogpost detailing 16.8 improvements.<\/p>\n

Developer Conversation<\/h2>\n

\"Image<\/a><\/p>\n

We spoke with Andrew Sage<\/strong> from Playground Games<\/strong> (Forza Horizon 4) and Dan Tunnell<\/strong> from Turn 10 Studios<\/strong> (Forza Motorsport) for some insights into how the improvements have affected their studios.<\/p>\n

Is the studio currently using Visual Studio 2019?<\/p>\n

\u201cWe have a mix of VS2017 and VS2019, depending on setup. Most engineers have moved to VS2019 for the main project, but we still have some tools projects that haven\u2019t been updated yet and are still using 2017.\u201d \u2013 Andrew<\/p><\/blockquote>\n

 <\/p>\n

\u201cWe just moved the whole studio to Visual Studio 2019 in the last week of January 2021.\u00a0 We’ve had one of our development branches running it since before the holidays but it just reached broad adoption a couple of days ago.\u201d \u2013 Dan<\/p><\/blockquote>\n

 <\/p>\n

What were some immediate benefits that you saw from faster link times?<\/p>\n

\u201cReducing iteration time to build and test changes allows us to be more productive, especially as gains add up over the course of all of the engineers multiple times per day. The main benefit was the removal of stalls when debugging with fast link when switching to full link, as these would add 30-45 seconds per debug session, which can be repeated multiple times per build.\u201d \u2013 Andrew<\/p><\/blockquote>\n

 <\/p>\n

\u201cThe biggest benefit I see is improved iteration time.\u00a0 Even if we only shave 30 seconds off link time, multiplying that by the number of devs in the studio and the number of times they compile each day becomes a large number really quick.\u201d \u2013 Dan<\/p><\/blockquote>\n

 <\/p>\n

What are some potential long-term benefits?<\/p>\n

\u201cLonger term, we will also be moving all our automated build servers over to Visual Studio 2019 to reduce our build validation and test turnaround times. This means a reduction in time from checking in code to knowing that the build has succeeded. Also, there will be less changes per build as these builds run continuously. Moving all our tools projects over to Visual Studio 2019 in the future would lead to a further productivity benefit for the tools team, and their ability to spend more time supporting our content creators.\u201d \u2013 Andrew<\/p><\/blockquote>\n

 <\/p>\n

\u201cOne of the longer-term benefits is reduced build times on our server farm.\u00a0 If someone checks in code that breaks the build, the faster we catch it the less downtime we experience.\u00a0 It also gives us faster tooling iteration since the time between a developer checking in new tooling code and the build system checking in new tools binaries is reduced.\u00a0 Additionally, there’s reduced time for “validation” builds.\u00a0 Prior to checking in, each developer builds a representative sample of the game binaries to ensure that they haven’t broken anything.\u00a0 This slows iteration (I’m sure you can see a pattern emerging – iteration time is king) and anything we can do to speed that up is extremely valuable.\u201d \u2013 Dan<\/p><\/blockquote>\n

 <\/p>\n

How might the link time improvements change the workflow patterns of the studio?<\/p>\n

\u201cBesides the improvement in focusing on fixing issues rather than waiting for builds to complete across the team, I think we will be able to focus on other productivity gains which then become a larger overall proportion of our processes. For instance, reducing dependencies in our codebase can improve build times even with the faster toolset and distributed build systems, and we will continue to focus our efforts in this area in the future. Also, using full link allows for less mental \u2018task switching\u2019 overhead caused by the stalls, as often when someone hits a stall they might swap to another task for a bit and then back in later, which can cause workflow to become interrupted.\u201d \u2013 Andrew<\/p><\/blockquote>\n

 <\/p>\n

\u201cIf I were to guess, I’d say that we’ll see less build breaks.\u00a0 Validation builds take a significant amount of time to complete.\u00a0 Even though they’re required, sometimes developers will omit that step if they’re in a hurry and feel confident that their fix is trivial enough to be safe.\u00a0 The faster the build times are the lower that barrier to entry is and the more likely people will perform that step even for changes they consider to be trivial and\/or safe.\u00a0 I also think we might see less code changes between compilations.\u00a0 The longer compilation and linking takes the more code a developer will add before they hit the button to compile.\u00a0 If compilation and linking is almost instantaneous, most developers will use that as a quick sanity check to verify that the method\/function\/class\/etc. that they just added compiles before moving on.\u201d \u2013 Dan<\/p><\/blockquote>\n

Upgrade Visual Studio<\/h2>\n

Download the latest Visual Studio 2019 Preview<\/a> and experience faster end-to-end build iteration times for your own projects.<\/p>\n

How will these improvements impact your day-to-day workflow? Talk to us in the comments below, on Twitter (@VisualC)<\/a>, or via email at visualcpp@microsoft.com<\/a><\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

The C++ team at Visual Studio has delivered substantial compile and link time improvements throughout Visual Studio 2019. Playground Games and Turn 10 Studios shares their link time improvement wins.<\/p>\n","protected":false},"author":39811,"featured_media":27478,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1,3927,218],"tags":[77,3884,140,3885,331,110,3883,3887,25,3886],"class_list":["post-27464","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cplusplus","category-game-development","category-performance","tag-build-throughput","tag-build-time","tag-c","tag-compile-time","tag-game-development","tag-games","tag-gaming","tag-iteration-time","tag-linker","tag-video-games"],"acf":[],"blog_post_summary":"

The C++ team at Visual Studio has delivered substantial compile and link time improvements throughout Visual Studio 2019. Playground Games and Turn 10 Studios shares their link time improvement wins.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts\/27464","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/users\/39811"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/comments?post=27464"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts\/27464\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/media\/27478"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/media?parent=27464"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/categories?post=27464"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/tags?post=27464"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}