Follow up On Feedback From Beta1 and DevLab About VC Project Conversion’s Performance
My name is Bogdan Mihalcea and I’m a developer on the C++ Project & Build team. In the last 2 years I worked on a new C++ project system build on top MSBuild.
I’m writing this blog to share some excellent news related to improvements we made to the performance of project conversion since the Beta1 build. This was possible because of the feedback we got from you, and I want to thank you for doing so!
During our development milestones, we had tests reporting that we have a very slow conversion for specific types of projects. They were usually containing many files (1000+) and they had a significant percentage of them containing file level configurations. After we analyzed the root cause and the costs of fixing, we made the assumption that these projects with many file level configurations were not very common, so the impact of the performance in this scenario would be low. Another assumption we made was that the conversion it is a onetime thing so the impact even if big for those rare cases it is a onetime tax. Based on these assumptions we prioritized the fix lower and decided to invest our time in making the performance of main line scenarios better.
Our team has a customer plan which includes the early release of the product through Beta1 and having various face to face meetings with customers. One of these events is the Visual C++ DevLab, which last time we kept in May 09. During this event we noticed that 20% of the customers were experiencing a very slow conversion.
We investigated and we realized that all cases were caused by less than 5% of the projects in the solution, which took the majority of conversion time. One common detail about those few projects is that they were very heavy on file level configurations. From the discussions with the customers we realized that it is unfortunately easy to get in that state due to various reasons (evolutionary codebases, easy to cause human error due to bulk select, bugs in previous conversions, not a easy bulk way to revert the file level configurations to project level).
That pretty much invalidated our assumption that it is not common to get in a situation where a customer can have a project with 1000+ files and more than half have file level configurations. The first assumption was also invalidated from the discussions we had with the customers and from feedback we received from Beta1’s customers, which revealed that even if the projects that have this case are not very common the unfortunate thing is that it is common for customers with hundreds of projects have 1 or 2 of these and that will impact the whole experience of the conversion for a much bigger percentage of the customers, than what we estimated earlier.
Furthermore the second assumption was challenged as well as we learnt from your feedback that conversion might not be a onetime tax as most of the companies prefer to stay for months in a dual state with the development done in parallel on both products and it is common to maintain only the previous version files as a baseline for changes and to reconvert always when the new product is used. Considering the new feature we added in Dev10, native multitargeting, this seems to make this approach even more popular.
After we got all this feedback we acted on it. We analyzed the code paths and we found places where we could optimize for this type of scenarios and made the fix. Below is an extract from our measurements before and after the fix.
% files in vcproj with FileConfigurations
Previous Conversion Time
Current Conversion Time
less than 10%
The common case projects (PerfLab test), with only few file level configurations, were impacted too as you can see from the above table. We used pristine lab machines dedicated to performance runs and the results are specific numbers to these machines.
We are constantly on lookout for feedback and we are adjusting our priorities based on it, and we will try to keep you guys up to date with the progress we make.
C++ Project & Build