{"id":23189,"date":"2010-04-23T20:01:00","date_gmt":"2010-04-23T20:01:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/maoni\/2010\/04\/23\/debugging-with-the-right-tools\/"},"modified":"2021-10-04T13:04:56","modified_gmt":"2021-10-04T20:04:56","slug":"debugging-with-the-right-tools","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/debugging-with-the-right-tools\/","title":{"rendered":"Debugging with the Right Tools"},"content":{"rendered":"

Wow, it\u2019s been almost a year since I last blogged <\/FONT>J<\/SPAN><\/SPAN> We just shipped CLR V4.0. Yay!<\/FONT><\/FONT><\/P>\n

 <\/FONT><\/P>\n

An internal email thread prompted me to write this blog entry \u2013 one very powerful tool I wanted to point out when you need to debug\/investigate issues is your debugger (if your debugger is also windbg\/cdb that is <\/FONT>J<\/SPAN><\/SPAN> since that\u2019s what I use and that\u2019s what I will talk about in this article). <\/FONT><\/FONT><\/P>\n

 <\/FONT><\/P>\n

For those of you who are interested in investigating memory related issues at all, whether it\u2019s because you don\u2019t like your current app\u2019s memory usage or you simply want to improve, learning to use the debugger is invaluable. If you haven\u2019t started using windbg\/cdb, I would strongly encourage you to \u2013 I promise you won\u2019t regret it.<\/FONT><\/P>\n

 <\/FONT><\/P>\n

I\u2019ve talked about using <\/FONT>SoS<\/FONT><\/A> before. We\u2019ve added some more SoS commands in V4.0 and some of those are related to GC like !AnalyzeOOM, !GCWhere and !FindRoots. You can read about them on the MSDN page. But I wanted to talk about some techniques that may not be obvious from reading reference material. <\/FONT><\/P>\n

 <\/FONT><\/P>\n

What do you want to know about GC when your program is? 2 of the most common things are <\/FONT><\/P>\n

 <\/FONT><\/P>\n

1) Why are GCs being triggered? <\/FONT><\/P>\n

2) Why do GCs take this much time?<\/FONT><\/P>\n

 <\/FONT><\/P>\n

And the answers are, respectively:<\/FONT><\/P>\n

 <\/FONT><\/P>\n

1) The difference of the heap before and after a GC<\/FONT><\/P>\n

2) The survivors from that GC<\/FONT><\/P>\n

 <\/FONT><\/P>\n

Let me explain those in detail. Each generation has its own allocation budget which I explained <\/FONT>here<\/FONT><\/A>. It\u2019s a value that we set and when exceeded we\u2019d want to trigger a GC. If we do a collection and discovered that there\u2019s a lot of memory survived we would set this allocation budget very big which means you\u2019d need to allocate more into that generation for us to want to trigger a GC. The rationale is that then we\u2019d have a chance to reclaim some sizeable space back next time we do a GC. Otherwise we\u2019d do all this work for a GC and not be able to find much dead memory. <\/FONT><\/P>\n

 <\/FONT><\/P>\n

So if you do a !dumpheap right before a GC you see what objects you have; then you do a !dumpheap right after that GC you see some of those objects disappeared. It\u2019s those objects that triggered this GC. Why? \u2018cause if we didn\u2019t have any objects disappearing, we wouldn\u2019t be doing GC (\u2018cause we\u2019d get no dead space back). When you hear people say \u201cyou are doing gen2 GCs because you are churning gen2\u201d this is preciously the explanation for that.<\/FONT><\/P>\n

 <\/FONT><\/P>\n

As of now, the CLR GC doesn\u2019t compact the LOH. So if you want to look at the LOH you can see exactly which parts of the memory disappeared. Here\u2019s a sample output of a heap range before and after a gen2 GC. Before the gen2 GC (I formatted the !dumpheap with the methodtable pointer replaced with their readable names):<\/FONT><\/P>\n

 <\/FONT><\/P>\n

Address              <\/SPAN>MethodTable          <\/SPAN>Size<\/SPAN><\/P>\n

————————————————–<\/SPAN><\/P>\n

00000007f7ffe1e0     <\/SPAN>System.Byte[] <\/SPAN>       <\/SPAN>2,080,000<\/SPAN><\/P>\n

00000007f81f9ee0     <\/SPAN>System.String <\/SPAN>       <\/SPAN>100,000<\/SPAN><\/P>\n

00000007f8212580     <\/SPAN>System.Byte[] <\/SPAN>       <\/SPAN>140,000<\/SPAN><\/P>\n

 <\/FONT><\/P>\n

After this gen2 GC for the same heap range:<\/FONT><\/P>\n

 <\/FONT><\/P>\n

Address              <\/SPAN>MethodTable          <\/SPAN>Size<\/SPAN><\/P>\n

————————————————–<\/SPAN><\/P>\n

00000007f7ffe1e0     <\/SPAN>Free        <\/SPAN>  <\/SPAN>       <\/SPAN>2,320,000<\/SPAN><\/P>\n

 <\/FONT><\/P>\n

So those 3 objects were collected. As the application author those objects may very well ring a bell for you why they were allocated so you can go from there see if\/how you can do some optimizations.<\/FONT><\/P>\n

 <\/FONT><\/P>\n

The survivors from the collection are what you see in the output of !dumpheap at the end of that GC.<\/FONT><\/P>\n

 <\/FONT><\/P>\n

In this <\/FONT>MSDN article<\/FONT><\/A> I talked about how to set a breakpoint at the end of a gen2 GC. To set a bp at the beginning of a gen2 GC simply replace RestartEE with SuspendEE. If you want to do this for gen0 GCs, replace 2 with 0. Note that this applies to only non concurrent\/background GCs though since concurrent\/background GCs would not have the EE suspended for the most part. <\/FONT><\/P>\n

 <\/FONT><\/P>\n

Often I see people quickly go search for some other tools when they really could whip out the debugger and set a few breakpoints to see what things look like. Of course there are many useful tools out there \u2013 and they tend to aggregate info for you so you don\u2019t have to do that yourself \u2013 but sometimes it\u2019s so convenient to just use the most direct and handy tool which is your debugger. <\/FONT><\/P>\n

 <\/FONT><\/P>\n

Another classic example is when people ask me for help with debugging memory leaks and the easiest and quickest solution is to set a conditional bp on kernel32!VirtualAlloc for native memory leaks. I\u2019ve seen this over and over again and this is how the conversation would usually go:<\/FONT><\/P>\n

 <\/FONT><\/P>\n

someone<\/SPAN>, \u201cMaoni, I have a managed app that\u2019s leaking memory. Can you help me with it?\u201d <\/SPAN><\/FONT><\/FONT><\/P>\n

me<\/SPAN>, \u201cIs it a managed memory leak?\u201d<\/SPAN><\/FONT><\/FONT><\/P>\n

someone<\/SPAN>, \u201cI don\u2019t know.\u201d<\/SPAN><\/FONT><\/FONT><\/P>\n

me<\/SPAN>, \u201cHow big is your managed heap? Can you do the SoS !dumpheap command?\u201d<\/SPAN><\/FONT><\/FONT><\/P>\n

someone<\/SPAN>, \u201cit\u2019s only X MBs\u201d [but his whole process uses much more than that]<\/SPAN><\/FONT><\/FONT><\/P>\n

me<\/SPAN>, \u201cDoes it grow? If you run your app for a while does the managed heap size change?\u201d<\/SPAN><\/FONT><\/FONT><\/P>\n

someone<\/SPAN>, \u201cNo\u201d (or \u201cNot much, not nearly as much as the whole memory usage increases\u201d)<\/SPAN><\/FONT><\/FONT><\/P>\n

me<\/SPAN>, \u201cCan you take a look at your virtual memory address space by doing !address?\u201d [!address gives you info on each virutal memory chunk. It\u2019s also a windbg\/cdb debugger extension but it\u2019s in ext.dll  <\/SPAN>which comes with the debugger package.]<\/SPAN><\/FONT><\/FONT><\/P>\n

someone<\/SPAN>, \u201cIt looks like there\u2019re a lot of chunks of size S\u2026\u201d<\/SPAN><\/FONT><\/FONT><\/P>\n

me<\/SPAN>, \u201cCan you set a conditional bp on kernel32!VirtualAlloc when the allocate size is S?\u201d [Which means: bp kernel32!virtualalloc “.if (@edx==S) {kp} .else {g}”]<\/SPAN><\/FONT><\/FONT><\/P>\n

[some time elapses]<\/FONT><\/FONT><\/SPAN><\/P>\n

someone<\/SPAN>, \u201cIt looks like they all have the same callstack!!!\u201d<\/SPAN><\/FONT><\/FONT><\/P>\n

 <\/FONT><\/P>\n

At this point it\u2019s clear what causes their memory leak.<\/FONT><\/P>\n

 <\/FONT><\/P>\n

If you need to see continuous output of some commands without disturbing the process too much you can always log it to a file (.logopen c:\\mydebugsession.txt) and analysis the log after the process runs for a while.<\/FONT><\/P>\n

 <\/FONT><\/P>\n

As I mentioned there are tools that will aggregate this info for you (eg, the DebugDiag tool you can find on microsoft.com which shows you the aggregated callstack from all the VirtualAlloc calls) but if you\u2019ve never used those tools it may take a while to set them up and learn how to use them. The debugger is something you have right there and use all the time so it\u2019s something you can fairly conveniently solve with a debugger I tend to use that.<\/FONT><\/P>\n

<\/FONT> <\/P><\/p>\n","protected":false},"excerpt":{"rendered":"

Wow, it\u2019s been almost a year since I last blogged J We just shipped CLR V4.0. Yay!   An internal email thread prompted me to write this blog entry \u2013 one very powerful tool I wanted to point out when you need to debug\/investigate issues is your debugger (if your debugger is also windbg\/cdb that […]<\/p>\n","protected":false},"author":3542,"featured_media":58792,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[685],"tags":[3011],"class_list":["post-23189","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet","tag-maoniposts"],"acf":[],"blog_post_summary":"

Wow, it\u2019s been almost a year since I last blogged J We just shipped CLR V4.0. Yay!   An internal email thread prompted me to write this blog entry \u2013 one very powerful tool I wanted to point out when you need to debug\/investigate issues is your debugger (if your debugger is also windbg\/cdb that […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/23189","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/3542"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=23189"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/23189\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/58792"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=23189"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=23189"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=23189"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}