{"id":10855,"date":"2016-10-05T11:41:07","date_gmt":"2016-10-05T18:41:07","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/vcblog\/?p=10855"},"modified":"2019-02-18T18:04:39","modified_gmt":"2019-02-18T18:04:39","slug":"faster-c-solution-load-with-vs-15","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cppblog\/faster-c-solution-load-with-vs-15\/","title":{"rendered":"Faster C++ solution load with VS 2017"},"content":{"rendered":"

The Visual C++ product has had projects ever since its inception.\u00a0 Visual C++ had its own IDE up through Visual Studio 6.\u00a0 Starting in Visual Studio .NET, C++ moved to a new IDE shared by Visual Basic, C#, C++, and other tools.\u00a0 This new IDE used COM everywhere and items in the Solution Explorer were based on IVsHierarchy<\/a>.\u00a0 Much of the existing project system was kept intact but it was exposed through a new set of COM objects that implemented IVsHierarchy and other interfaces.<\/p>\n

In Visual Studio 2010, the underlying project\/build system was changed<\/a>from VCBuild to MSBuild.\u00a0 This is also when the file extension changed from .vcproj to. vcxproj.\u00a0 At this point, the existing set of (relatively new) COM objects was kept, but forwarded into a new set of managed \u201cshim\u201d objects that then called into the new project system.\u00a0 This new project system eventually became known as the Visual Studio Project System (VSPS).<\/p>\n

As years went by, developers created ever larger solutions and projects.\u00a0 This created a problem as Visual Studio took longer and longer to load these ever-increasing solutions.\u00a0 Each project would get parsed and a large graph of objects would be created in memory and kept there.\u00a0 At least one object per node in the hierarchy would get created.\u00a0 This approach takes a lot of time and uses a lot of memory.\u00a0 Visual Studio tried to keep up and has implemented various strategies to support large projects.\u00a0 For instance, you can explicitly unload projects to prevent them from getting loaded and ASL (Asynchronous Solution Load<\/a>) was introduced in Visual Studio 2013.\u00a0 ASL tries to load most projects on a background thread and make the UI immediately responsive for other tasks.<\/p>\n

While ASL works to a large extent, we have heard from many developers that they prefer to just wait for the whole solution to finish loading and all other background work to finish before trying to use the IDE.\u00a0 You know, open your solution and go get coffee.<\/p>\n

Even after all of the projects are loaded, there is still other stuff happening that is using CPU and disk.\u00a0 In the status bar, you are probably used to seeing the following:<\/p>\n

\"initializingprojects\"<\/a><\/p>\n

This message means we are doing what we call a \u201cdesign-time build\u201d.\u00a0 The VSPS is evaluating projects and figuring out what would be built and how it would be built.\u00a0 We do this to generate command-lines for every file in every project and then compare them to what we have stored in the browse database<\/a>.\u00a0 If they have changed, we write them into the database.\u00a0 The command lines are used for Intellisense and also to resolve include files that are indexed in the browse database.<\/p>\n

After this \u201cinitializing\u201d phase you will see that we are checking for out-of-date files (and then updating them if needed).\u00a0 Finally, the status bar shows ready.\u00a0 However, if you take a look in Task Manager, you will notice that one CPU is still getting heavily used.\u00a0 In the background, we are still doing work.\u00a0 The first task is populating the \u201cexternal dependencies\u201d node of each project.\u00a0 This node contains files that aren\u2019t explicitly in the project but are #included directly or indirectly by some file in the project.\u00a0 After this step, there is one more invisible step which checks the database for orphaned records, such as files we indexed that aren\u2019t used anymore (directly or indirectly) by any project.\u00a0 All of this happens every time you open a solution even if absolutely nothing has changed since the last time you used it.<\/p>\n

Let\u2019s take a look at loading the Chromium solution and see how long each of these steps is taking in Visual Studio 2015 Update 3 vs. Visual Studio 15 Preview 5<\/a>.\u00a0 If you follow the instructions on the Chromium website<\/a>, the solution is generated by the command \u201cgn gen –ide=vs out\\Default\u201d.\u00a0 This results in ~4600 projects of which ~1000 are \u201cSolution Folders\u201d and the others are .vcxproj projects.<\/p>\n

\"picture\"<\/a><\/p>\n

The following results are on Windows 10 from my personal machine which is an Intel Core i7-4790 @ 3.6GHz and two SSDs: one as system drive and one for source code (Samsung 850 Pro).\u00a0 The first set of results is in Visual Studio 2015 Update 3 and the second set is from Preview 5 of Visual Studio 15<\/a>.<\/p>\n

We realize that the \u201cparsing\/resolving includes\u201d phase is about 25% slower than in VS2015 Update 3.\u00a0 We are working on improving this as well and expect to get that resolved soon. <\/em><\/p>\n

Why is solution load with Visual Studio \u201c15\u201d so much faster?<\/strong><\/h4>\n

Because of our existing layered architecture, it was relatively easy to insert a new caching layer that can answer many questions about projects and configurations without having to actually load the project into the VSPS.\u00a0 This new layer uses a SQLite database to quickly retrieve information on demand.<\/p>\n

When a solution is loaded and we are asked to load a C\/C++ project, we get a list of all .vcxproj files this solution will load.\u00a0 We check the database to see if we already have these projects and if any files have changed.\u00a0 If we need to update the information about a set of projects, those projects are put into a queue.\u00a0 That queue is handled by multiple external processes that use MSBuild to load the projects and collect information about them and write it to the database.<\/p>\n

As we are asked to load each project, we create a set of small shim objects that can service many requests without needing to fully load a project.\u00a0 We can provide all of the information that our Intellisense engine needs, as well as provide what Solution Explorer needs through information in the database.\u00a0 If an API is called that needs a real project (such as modifying project settings), the underlying shim will load the project on the fly and delegate to it.<\/p>\n

Because of this change the load time of individual projects went way down, but not as low as we desired.\u00a0 Profiling revealed some pretty bad algorithms with N^2 time complexity in various places.\u00a0 Our memory use also dramatically dropped after this change, but we also found some pretty bad memory use inside our own objects.\u00a0 We were able to trim the size of each object that represents a file in a solution (including external dependencies) from 120 bytes to 44 bytes per instance.\u00a0 It may not seem like much, but some large solutions end up with millions of these objects.\u00a0 We are still working on improving the performance of project load and I expect to see some additional improvements before shipping the final version.<\/p>\n

This feature is truly experimental<\/em>, there are still some issues with fast project load we want you to be aware of.<\/p>\n

    \n
  1. Projects that need upgrading should be upgraded first before trying to use fast project load on them as upgrade will not happen during FPL.<\/li>\n
  2. Our story today for being able to build is not complete for this preview release. Very simple solutions with one project might build but that’s pretty much it.<\/li>\n
  3. Projects will load through VSPS on demand such as when explicitly editing a project (e.g. adding files or changing properties).\u00a0 A large\nproject may take a few seconds to load.\u00a0 We want to signal this to the user but we haven\u2019t yet in all cases.<\/li>\n
  4. Third party plugins could choose to walk the entire solution hierarchy asking for properties that causes all projects to be fully loaded in VSPS, effectively defeating any benefits of FPL.<\/li>\n<\/ol>\n

    The IDE\u2019s Output Window will display a message whenever a project is fully loaded into the VSPS.\u00a0 Please let us know if you see these messages unexpectedly.<\/p>\n

    Lightweight Solution Load<\/strong><\/h4>\n

    There is another experimental effort underway in Visual Studio to improve solution load called \u201clightweight solution load\u201d.\u00a0You can enable this feature\u00a0using the following option.<\/p>\n

    \"fpl\"<\/a><\/p>\n

    This is a completely different approach and you can read about it on the Visual Studio blog\u00a0here<\/a>.\u00a0 Generally, it will avoid loading projects at all and will only load a project when a user explicitly expands a project in Solution Explorer.\u00a0 The C++ team has been focused on Fast Project Load and so our support for lightweight solution load is currently minimal.\u00a0 In the RC \u00a0release of Visual Studio 15<\/a>, we expect to support FPL in conjunction with Lightweight Solution Load.\u00a0 This combination should provide a great experience.<\/p>\n

    Wrap Up<\/strong><\/h4>\n

    As always, we welcome your feedback and we would love to learn from your experiences as you try these features out. Do let us know if you run into any issues trying out faster solution load through report a problem tool<\/a>.<\/p>\n

    You can also email <\/a>us your query or feedback if you choose to interact with us directly! For new feature suggestions, let us know through User Voice<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

    The Visual C++ product has had projects ever since its inception.\u00a0 Visual C++ had its own IDE up through Visual Studio 6.\u00a0 Starting in Visual Studio .NET, C++ moved to a new IDE shared by Visual Basic, C#, C++, and other tools.\u00a0 This new IDE used COM everywhere and items in the Solution Explorer were […]<\/p>\n","protected":false},"author":265,"featured_media":35994,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[270,217,218],"tags":[343,344],"class_list":["post-10855","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-announcement","category-faster","category-performance","tag-asl","tag-fast-solution-load"],"acf":[],"blog_post_summary":"

    The Visual C++ product has had projects ever since its inception.\u00a0 Visual C++ had its own IDE up through Visual Studio 6.\u00a0 Starting in Visual Studio .NET, C++ moved to a new IDE shared by Visual Basic, C#, C++, and other tools.\u00a0 This new IDE used COM everywhere and items in the Solution Explorer were […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts\/10855","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/users\/265"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/comments?post=10855"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts\/10855\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/media\/35994"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/media?parent=10855"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/categories?post=10855"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/tags?post=10855"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}