February 4th, 2021

Working with large .NET 5 solutions in Visual Studio 2019 16.8

Paul Vick
Principal Software Engineer

With the release of .NET 5, migration of solutions from .NET Framework has increased. In particular, we have started to see very large solutions being moved. To ensure this experience is as good as possible, we have been working on optimizing Visual Studio to handle solutions that contain large numbers of .NET 5 and .NET Core projects. Many of these optimizations were shipped in the 16.8 release, and this blog post walks through the improvements we’ve made.

Running the C# and VB compiler out of process

Roslyn, the C# and Visual Basic compiler, parses and analyzes the entire solution to power services such as IntelliSense, Go to Definition, and diagnostics/errors. As a result, Roslyn tends to consume resources that increase proportionally with the size of the open solution, which can get quite significant for large solutions. Roslyn has already worked to minimize this impact by aggressively caching on disk information that isn’t immediately required, but even with that caching, it cannot escape the need to keep data in memory.

To reduce its impact on larger Visual Studio solutions, the Roslyn team has spent considerable effort moving the Roslyn compiler out of the Visual Studio process and into its own process. Running Roslyn in its own process frees up resources within Visual Studio itself and allows the Roslyn compiler more room to do its work. For large solutions, this can save up to a third of the memory consumed by Visual Studio when you open a large solution.

Streamlining the Dependencies node

Every .NET 5 and .NET Core project has a node in the Solution Explorer named “Dependencies” that displays all the things that the project depends on: other projects, assemblies, NuGet packages, etc. In addition to showing the immediate dependencies of the project, the node also shows the transitive dependencies of the project, i.e., all the things each dependency itself depends on, and so on. With a project of any reasonable size, this list of transitive dependencies can get quite large.

Unfortunately, the original implementation of the “Dependencies” node was not particularly efficient in the way that it stored the transitive dependency information in memory. It held on to much more information than was needed, and much of the data was redundant. We rewrote the code to keep only information that was absolutely required, and we started piggybacking on the existing information on dependencies already kept by NuGet. This rewrite saved up to 10-15% of the memory consumed by Visual Studio when you open a large solution.

Reducing duplicate information in MSBuild

After Roslyn, one of the other major consumers of resources in a Visual Studio process is MSBuild. This is because, as the build engine, much of the IDE experience is powered by MSBuild’s object model. Although we’ve made project files themselves much smaller in .NET 5 and .NET Core, there are a significant number of supporting project files that get imported into projects through the SDK. Evaluating all those files is necessary to understand and build a project, and can consume up to another third of the memory consumed by Visual Studio when you open a large solution.

Although project files generate a lot of data, much of that data is repetitive, and we have started de-duplicating that data in memory. Strings are one of the most common byproducts of the project system, and they store information like file names, options, and paths. Paths, specifically, can be quite long and can end up consuming a lot of memory if there are too many of them, or if they are duplicated too many times. Ensuring that we keep only one copy of a string saved up to 5-10% of the memory consumed by Visual Studio when you open a large solution.

Reducing project copies held onto by the project system

One of the important design aspects of the .NET 5 and .NET Core project system is asynchrony. Often the project system needs to do work in response to a user action (for example, adding a new reference to a project). Instead of blocking Visual Studio while it finishes its work, the project system allows the user to continue working in the IDE and does the work in the background.

Because the user can continue to make changes to a project (say, adding yet another reference) while the project system is processing previous changes in the background, the project system must save snapshots of the project data to ensure that a later action doesn’t conflict with an earlier one. As a result, the project system can easily end up with multiple copies of a project’s data in memory at once. If the project system is not careful about managing these copies, they may be held on to longer than needed, or even leaked and retained permanently. We’ve worked aggressively to reduce the number of copies we hold on to at one time.

Large solution load improvements

With all this work, we have significantly optimized the experience of working with large .NET 5 and .NET Core solutions. As of 16.8, in many of our tests we have seen a 2.5x improvement in the size of solution we can open before running into resource issues. We have also seen a decrease of up to 25% in crashes reported due to resource exhaustion.

The improvements listed above are just the beginning of the changes we are making to improve the experience of working with large solutions in Visual Studio. Individual solution performance still may vary, depending on the size of the solution, type of projects, extensions loaded, etc., and there are still areas that we are looking at to improve. We encourage any users who are experiencing issues with slowness or crashes loading solutions to contact us at vssolutionload@microsoft.com so we can continue to improve the solution load experience for all solutions!

Author

Paul Vick
Principal Software Engineer

Paul Vick is a principal software engineer in Visual Studio. He has worked extensively on databases, programming languages, and application performance. He has also written about these subjects, including a specification for Visual Basic, a book on Visual Basic, and a weblog.

27 comments

Discussion is closed. Login to edit/delete existing comments.

  • Jason Brower

    I am really excited to see an effort to improve the performance of Visual Studio and large solution processing. I often find myself working with 125-150 DLL solutions that have never been easy to remain focused on my work, because Visual Studio has had so many issues.

    I have tried to switch away to other tools, but always come crawling back.

    Keep up the good work.

  • Ion Sorin Torjo

    All this crap would go away, if you actually made Visual Studio 64 bit. I simply can’t understand why you resist this so much.
    People have been asking for this for ages, and you keep on doing just workarounds.
    At some point, everyone will simply migrate to Rider (the only reason I haven’t done this yet is because it doesn’t support UWP debugging yet)

  • Rocco Balzamà

    Good morning,
    we are very interested in new developments in VisualStudio 2019,
    but before making a purchase, it would be possible to know when and if the new version “VisualStudio 2021” will be released
    Thanks

  • Andriy Savin · Edited

    Hi, while this all work is great, VS is still a pretty hungry on resources. I noticed that even with relatively small solutions when several VS instances are launched many other apps' UI becomes quite slow. This is much more observable on WinForms/GDI applications, but for example windows file explorer becomes slow as well. I noticed with help of Spy++ that VS generates huge amount of windows messages even when it's idle. This includes WM_TIMER...

    Read more
    • Paul VickMicrosoft employee Author

      Thanks for the feedback, Andriy. Have you submitted an issue through the VS feedback tool?

      • Simon Felix

        I appreciate that 500+ project solutions open faster and all the improvements that have been made.

        I usually work with much smaller solutions (1-5 projects) on a Surface Pro with 8GB mem. For this case, the performance of VS is not ideal. I haven’t posted an issue (because I’ve seen too many reports die a quiet death.)

      • Andriy Savin

        Yes, here https://developercommunity.visualstudio.com/content/problem/799943/visual-studio-noticeably-slows-down-the-whole-syst.html
        Note that in that report I was referring to a VM environment. Since then, mostly because of this problem, I switched from VM to "local" environment, i.e. running OS and applications directly on the hardware. However, the problem remains, it's just somewhat less annoying than on a VM.
        The net effect for me is that no matter how powerful and expensive my PC is, it's perceived performance becomes poor even with pretty...

        Read more
      • Jorge Morales Vidal

        You may want to create a new case if required and add the new details about your physical computer. I would like to upvote it as I saw a similar behavior but will lots of Visual Studio instances open and Windows File Explorer redraws itself a bit slower than usual. It is not a big deal but it is noticeable.

  • Nicolas Musset

    How about finally making Visual Studio 64-bit? A feature that has been requested by developers for more than a decade.

    Usually are available RAM is not the bottleneck, I casually use development machines with between 16 to 128 GB of RAM. Every other projects at Microsoft has moved to 64-bit, all the WinAPI is available in 64-bit. What could possibly be the low-hanging fruits in Visual Studio codebase that prevent anyone from making it happen for...

    Read more
    • Stevie White

      I am going to have to agree on this one. I really wish the VS team would stop hiding behind "Oh we're making things out of process to make it better" excuse or staying silent on this matter. And please stop trying to push us towards using some hodge-podge online / cloud solution. That isn't going to fly, especially if it requires us to get nickel and dimed out of Azure costs on top of...

      Read more
    • Daniel Smith

      Absolutely everyone agrees that Visual Studio needs to go 64-bit…with the exception of Microsoft.

    • Kirill Skrygan

      Same for us, we have around 650 projects in our solution.
      Luckily there is Rider so we have just skipped VS completely in our team and switched to Rider. It’s being awesome so far! 🙂

    • Aleksander Thomas Weingerl

      +1
      Agree!

      • Bill Eisenman

        +1 for me also. My development projects frequently run into 32 bit IDE VS limitations, ugh. Then I could use my 64-bit 64GB dev machine to its fullest for once.

      • Martin Sedlmair

        +1 I agree. 64bit is the way to go. Also on our side we are looking for alternatives (like Rider). Many of my team members have to kill VS approx. 5 times a day with the task manager because it simply hangs or crashes for itself. We only have around 250 projects in our solution. Fileing in issues and a trace report is pretty hard when you cannot do it while VS stops working 😉

    • Hughes, Danial

      Completely agree. Move VS to 64bit already!!!!

  • Martin Ba

    How does this affect large C++ solutions? The stuff about the dependency node and de-duplication inside MSBuild sounds as if this would benefit C++ projects as well?

    We’re currently working with a 850 projects (approx 90% C++, rest C#) sln and VS 2019 16.7.6

    I mean, holy cow is this already waaay better than it was with VS2017, but it would be interesting if we can expect performance improvements moving from 16.7 to 16.8 ??

    • Paul VickMicrosoft employee Author

      The C++ project system is, unfortunately, mostly a separate codebase so many of these improvements don’t apply to C++ projects.

      • László Csöndes

        It’s 2021. Support 64 bit already.

  • Alfonso Ramos Gonzalez · Edited

    It isn't good enough. I was going to write a rant, but I decided to make a summary of the features I'd want instead:

    - Configure how many background tasks run concurrently.
    - Configure how many targets to run Roslyn against concurrently (always with priority to what is selected in the editor).
    - Show what the background tasks are doing (edit: in particular I would like to see if a particular rule is eating time).
    -...

    Read more
  • Frederic Forjan

    does the roslyn out of process also applies to .Net Framework project ? or only .net 5 ?

    • Paul VickMicrosoft employee Author

      That applies to all project types, so it also improves .NET Framework projects as well.

      • WALEED S

        From my personal experience I started with [ Visual Studio 2008 ], one by on i have used all others versions. until i used [ Visual Studio 2019 ] , then got shocked! [ Visual Studio 2019 ] it so fast and so powerful version

  • Shaaban Alqassas

    thanks

  • Farhan Ansari

    What are the numbers for large solutions which were tested?

    • Paul VickMicrosoft employee Author

      We were primarily testing on solutions that have about 1,000 projects in them.