{"id":5931,"date":"2009-12-08T06:51:27","date_gmt":"2009-12-08T06:51:27","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/bharry\/2009\/12\/08\/looking-at-virtual-memory-usage\/"},"modified":"2018-08-14T00:21:14","modified_gmt":"2018-08-14T00:21:14","slug":"looking-at-virtual-memory-usage","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/bharry\/looking-at-virtual-memory-usage\/","title":{"rendered":"Looking at Virtual Memory Usage"},"content":{"rendered":"

One of the big problems that I\u2019ve talked about is virtual memory exhaustion and the resulting VS instability.  Today, VS is a 32-bit process \u2013 I\u2019m sure that\u2019s going to change (become 64-bit) at some point but it won\u2019t for 2010.  A 32-bit process has 2GB of address space on a 32-bit OS.  You can throw the 3GB switch and get another GB at the cost of taking a GB away from the OS \u2013 it works OK but the OS doesn\u2019t always like it \u2013 particularly if you are running server components on your machine (like IIS).  If you are on a 64-bit OS, 32-bit processes get a full 4GB of virtual address space without any penalty to the OS \u2013 handy eh?<\/p>\n

In the end, many customers are still running on 32-bit OSes and asking them to go change boot parameters (the 3GB switch) just isn\u2019t an appropriate thing for us to do, so we\u2019re focusing on making sure VM works well in the 2GB of address space that many customers have available.<\/p>\n

This can be complicated by Virtual Address space fragmentation.  This is caused by smallish gaps between other virtual address space allocation.  You can reach the point where there\u2019s enough virtual address space left but there\u2019s not enough contiguous space to satisfy allocations.  So saying everything is OK as long as virtual memory allocation stays below 2GB really isn\u2019t realistic.  The app will become unstable well before that.<\/p>\n

Further, any test we do has to take into account that users will always do some things we didn\u2019t expect so running VM all the way up to the threshold and saying \u201cgood enough\u201d just isn\u2019t acceptable either.<\/p>\n

So what do you do?<\/p>\n

We\u2019ve started by gathering some \u201creal world apps\u201d.  By this I mean some of you, our customers, have actually given us copies of your large apps.  We\u2019re incredibly grateful for this because they are great tools to ensure that we are staying in touch with the kinds of apps our customers are building and how VS is working for them.  Oh, and we\u2019re using a couple of internal MS apps as well.  We\u2019ve chosen 6 of them (apps of various types: web, WPF, Sharepoint, etc) that are all large and we believe characterize a large portion of our customer base.<\/p>\n

We then choose a set of operations \u2013 load the solution, edit some code, design some forms, debug, checkin some changes, etc, etc that represent a realistic end user interaction with each application.<\/p>\n

And lastly, we\u2019ve chosen a threshold or 1.5GB of VM.  We will not consider it acceptable until all 6 of the scenarios are able to execute without ever going over 1.5GB of VM.  We believe this will allow adequate room for users to do things we didn\u2019t anticipate and still have a buffer before getting near the instability threshold.<\/p>\n

Here\u2019s a picture of where we currently stand with the 6 apps (btw, I\u2019ve changed the names of the apps to protect the identity of people we are working with).  Most of these apps were RED when we started the VM effort a couple of months ago.  As you can see we\u2019ve made some good progress but we aren\u2019t done yet.<\/p>\n

\"image\"<\/a> <\/p>\n<\/p>\n

Let\u2019s take a look at some detail on one of the \u201cbad ones\u201d.  Here you can see the way we are tracking this.  You can see the \u201csteps\u201d in the scenario, the virtual memory after each step and the comparison to VS 2008 SP1 (for all steps where there is an equivalence).  You\u2019ll see there are more bars here than steps listed, I think that\u2019s an artifact of the slideware (not all steps are listed).  We use this to look for any places where VM jumps significantly or varies substantially from VS 2008 to focus investigations.<\/p>\n

\"image\"<\/a> <\/p>\n

We also plot the VM growth from different builds against each other.  In this case, you can see that in the Nov 11th build (21111) the CB died halfway through the scenario because it ran out of VM and you can see that by Nov 25th (21125) we had improved it so that that not only did it complete but it is generally better than VS 2008 (let us remember that VS 2008 isn\u2019t flawless so we don\u2019t blindly take parity with it as success).  As you can see we still have a bit of work left to do to hit out 1.5GB goal.<\/p>\n

\"image\"<\/a> <\/p>\n

We also track improvements over time from build to build in more of a trend fashion:<\/p>\n

\"image\"<\/a> <\/p>\n<\/p>\n

And lastly we use a \u201cledger\u201d format for tracking fine grained progress.  This is the best way I\u2019ve found to track this kind of thing: Identify the goal and current status against it.  Identify the fixed currently in progress and the expected improvements (generally an informed guess) and identify the areas that are under investigation (the things that just don\u2019t look right) and best wild ass guess you can make about how much improvement \u201cought\u201d to be available based on what it is vs what you think it should be.  This gives you a way to see things moving through the pipeline, understand progress towards the goal and make priority trade-offs.<\/p>\n

\"image\"<\/a> <\/p>\n

Hopefully that gives you some insight into how we are thinking about the VM issue and the progress we are making.<\/p>\n

Brian<\/p>\n","protected":false},"excerpt":{"rendered":"

One of the big problems that I\u2019ve talked about is virtual memory exhaustion and the resulting VS instability.  Today, VS is a 32-bit process \u2013 I\u2019m sure that\u2019s going to change (become 64-bit) at some point but it won\u2019t for 2010.  A 32-bit process has 2GB of address space on a 32-bit OS.  You can […]<\/p>\n","protected":false},"author":244,"featured_media":14617,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[4],"class_list":["post-5931","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-visual-studio"],"acf":[],"blog_post_summary":"

One of the big problems that I\u2019ve talked about is virtual memory exhaustion and the resulting VS instability.  Today, VS is a 32-bit process \u2013 I\u2019m sure that\u2019s going to change (become 64-bit) at some point but it won\u2019t for 2010.  A 32-bit process has 2GB of address space on a 32-bit OS.  You can […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/posts\/5931","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/users\/244"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/comments?post=5931"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/posts\/5931\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/media\/14617"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/media?parent=5931"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/categories?post=5931"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/bharry\/wp-json\/wp\/v2\/tags?post=5931"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}