{"id":96195,"date":"2017-05-19T07:00:00","date_gmt":"2017-05-19T21:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/?p=96195"},"modified":"2019-03-13T01:11:42","modified_gmt":"2019-03-13T08:11:42","slug":"20170519-00","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20170519-00\/?p=96195","title":{"rendered":"Debugging a GDI resource leak: Case study"},"content":{"rendered":"

I was asked to help debug a problem. A program was leaking GDI bitmaps like crazy, and after a while, the GDI resource handle count reached 9,999, at which point GDI said, “That’s it, I’m cutting you off.” <\/p>\n

The problem isn’t discovered until after the limit has been reached. <\/p>\n

To debug this, I’m gong to use different poor man’s way of identifying memory leaks<\/a>. We begin the story with a GDI handle that has been identified as leaked: 0x13054e2f<\/code>. My first step is to get some basic information about this GDI object by simulating a call to Get­Object<\/code>. <\/p>\n

\n0:061> .dvalloc 1\nAllocated 1000 bytes starting at 00000000`03610000\n<\/pre>\n
First, I allocated some memory that I can use to hold the BITMAP<\/code> structure. <\/p>\n
\nntdll!DbgBreakPoint:\n00007ffb`65ef7570 int     3\n0:061> t\nntdll!DbgBreakPoint:\n00007ffb`65ef7571 ret\n0:061> r rip=gdi32!GetObjectW\n0:061> r r8=0x00000000`03610000\n0:061> r rdx=0x68\n0:061> r rcx=0x13054e2f\n0:061> r\nGDI32!GetObjectW:\n00007ff9`9658e2f0 mov     qword ptr [rsp+8],rbx\n<\/pre>\n
Next, I simulate a call to the Get­ObjectW<\/code> function<\/a>¹ by setting up the inbound parameter registers: rcx<\/code> is the GDI object we are geting information about, rdx<\/code> is the size of the output buffer, r8<\/code> is the output buffer itself (which we just allocated). <\/p>\n
I’m pulling a super-sneaky trick here. Normally, we reserve shadow space for the outbound call, and then simulate the call<\/cODE> instruction by pushing the return address on the stack. But we didn't do any of that here. We just moved rip<\/code> directly to the function we wanted to call. But I can skip those steps because I stepped to the ret<\/code> instruction. This means that the stack is already set up the way it would be immediately upon entry to the function. We are reusing the stack frame of the Dbg­Break­Point<\/code> function! We are reusing the shadow space provided by the caller, and we are taking advantage of the proper stack alignment established by the caller. <\/p>\n
\n0:061> gu\nntdll!DbgUiRemoteBreakin+0x34:\n00007ff9`9730f4f4 jmp     ntdll!DbgUiRemoteBreakin+0x36 (00007ff9`9730f4f6)\n0:061> r\nrax=0000000000000020 rbx=00007ff99730f4c0 rcx=00007ff9965b877a\nrdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000\nrip=00007ff99730f4f4 rsp=0000000007a7f7e0 rbp=0000000000000000\n r8=0000000007a7f7a8  r9=0000000000000000 r10=0000000000000000\nr11=0000000000000206 r12=0000000000000000 r13=0000000000000000\nr14=0000000000000000 r15=0000000000000000\niopl=0         nv up ei pl nz na pe nc\ncs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202\nntdll!DbgUiRemoteBreakin+0x34:\n00007ff9`9730f4f4 jmp     ntdll!DbgUiRemoteBreakin+0x36 (00007ff9`9730f4f6)\n0:061> dd 0x00000000`03610000 L8\n00000000`03610000  00000000 00000010 00000010 00000002\n00000000`03610010  00010001 fffff901 00000000 00000000\n0:061> dt contoso!BITMAP 0x00000000`03610000\n   +0x000 bmType           : 0n0\n   +0x004 bmWidth          : 0n16\n   +0x008 bmHeight         : 0n16\n   +0x00c bmWidthBytes     : 0n2\n   +0x010 bmPlanes         : 1\n   +0x012 bmBitsPixel      : 1\n   +0x018 bmBits           : (null)\n<\/pre>\n
I run the Get­ObjectW<\/code> function with the gu<\/code> command, which means \"go until the current function returns\". When it returns, I verify that the call succeeded (by checking the return value in rax<\/code>), and the dump the BITMAP<\/code> structure. <\/p>\n
This tells me that I have a 16×16 monochrome bitmap. Monochrome bitmaps are rarely-used in Windows nowadays. One place you'll see them is in icons, since an icon consists of two bitmaps: A monochrome mask and a color image. <\/p>\n
So let's assume that we're leaking the mask of an icon. These things come out of two functions: Get­Icon­Info<\/code> and Get­Icon­Info­Ex<\/code>, so I set breakpoints on both. (Actually three functions since Get­Icon­Info­Ex<\/code> has both Unicode and ANSI variations.) <\/p>\n
\n0:061> bp user32!geticoninfo\n0:061> bp user32!geticoninfoexw\n0:061> bp user32!geticoninfoexa\n0:061> g\n<\/pre>\n
The breakpoint hits pretty quickly. <\/p>\n
\nBreakpoint 0 hit\nUSER32!GetIconInfo:\n00007ff9`96cf1dd0 sub     rsp,38h\n0:005> kc3\nCall Site\nUSER32!GetIconInfo\ncontoso!CNotificationIcon::IsIconCorrectSize+0x42\ncontoso!CNotificationIcon::Modify::__l6::\n    <lambda_5a5c6bde8a112e005a026c6f34886eee>::operator()+0x1f\n<\/pre>\n
Going back to the source code for Is­Icon­Correct­Size<\/code> confirms that this function calls Get­Icon­Info<\/code> (presumably to get the size of the icon) and forgets to delete the two bitmaps. <\/p>\n
Root cause identified. The fix is to delete those bitmaps. <\/p>\n
Forgetting to delete the bitmaps that come out of the Get­Icon­Info<\/cODE> family of functions is a common mistake. <\/p>\n
¹ The ntsd<\/code> debugger doesn't have a C compiler built-in, so we have to build these things manually. <\/p>\n","protected":false},"excerpt":{"rendered":"
If it leaked once, it’ll leak again.<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-96195","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"
If it leaked once, it’ll leak again.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/96195","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=96195"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/96195\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=96195"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=96195"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=96195"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}