{"id":107233,"date":"2022-09-30T07:00:00","date_gmt":"2022-09-30T14:00:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/oldnewthing\/?p=107233"},"modified":"2022-09-30T07:17:25","modified_gmt":"2022-09-30T14:17:25","slug":"20220930-00","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20220930-00\/?p=107233","title":{"rendered":"Debugging coroutine handles: Looking for the source of a one-byte memory corruption"},"content":{"rendered":"<p>A team was chasing a one-byte use-after-free memory corruption bug. These bugs are really frustrating to chase down because the memory corruption typically doesn&#8217;t trigger an immediate crash, but rather results in a delayed crash, which means that the culprit has done the damage and run away long before the problem is detected.<\/p>\n<p>We join the debugging session already in progress. We have determined that the corruption is to a memory block that previously contained a coroutine frame at offset <code>0xc0<\/code>.<\/p>\n<p>The state machine of a coroutine exists in the <code>_ResumeCoro$2<\/code> function, so we can start there:<\/p>\n<pre>contoso!DoStuffLater$_ResumeCoro$2:\r\n    mov     r11,rsp\r\n    ... stack frame nonsense ...\r\n\r\n    mov     rsi,rcx                 \/\/ rsi = coroutine frame pointer\r\n    mov     [rsp+28h],rcx\r\n    movzx   eax,word ptr [rcx+8]    \/\/ eax = coroutine state\r\n    mov     [rsp+20h],ax\r\n    inc     ax                      \/\/ artificially add 1\r\n    cmp     ax,8\r\n    ja      contoso!DoStuffLater$_ResumeCoro$2+0x3e5\r\n    ja      00007ffc`e777a5b5       \/\/ invalid index, die (jump to int 3)\r\n\r\n    movsx   rax,ax\r\n    lea     rdx,[contoso!__ImageBase]\r\n    mov     ecx,[rdx+rax*4+1BA5E0h] \/\/ look up jump table RVA\r\n    add     rcx,rdx                 \/\/ convert to absolute address\r\n    jmp     rcx                     \/\/ jump there\r\n\r\ncontoso!DoStuffLater$_ResumeCoro$2+0x3e5:\r\n    int     3\r\n<\/pre>\n<p>We see from the disassembly that the jump table starts at relative offset <code>0x1ba5e0<\/code>. We won&#8217;t dig into the jump table yet; let&#8217;s see if we can find the corruption point, which is a single-byte corruption at offset <code>0xc0<\/code> from the start of the coroutine frame. Maybe we&#8217;ll be lucky and the access is directly into the frame.<\/p>\n<pre>0:026&gt; #c0h contoso!DoStuffLater$_ResumeCoro$2\r\ncontoso!DoStuffLater$_ResumeCoro$2+0x136:\r\n    mov     [rsi+0C0h],al\r\n<\/pre>\n<p>Oh my goodness, we found a single-byte write at offset <code>0xc0<\/code> in the coroutine frame! Let&#8217;s see who is doing it.<\/p>\n<pre>    mov     eax,6\r\n    mov     [rsi+8],ax\r\n\r\n    mov     rdx,rsi\r\n    mov     rcx,rbx\r\n    call    contoso!winrt::impl::notify_awaiter&lt;`winrt::resume_foreground'::`2'::awaitable&gt;::\r\n            await_suspend&lt;std::experimental::coroutine_traits&lt;winrt::fire_and_forget&gt;::promise_type&gt;\r\n    mov     [rsi+0C0h],al   \/\/ WRITE HAPPENS HERE\r\n<\/pre>\n<p>The first two instructions set the coroutine state to 6, which happens as part of coroutine suspension.<\/p>\n<p>The second group of instructions call the <code>await_<wbr \/>suspend<\/code> for a <code>resume_foreground<\/code> awaiter. This is in code that is moving forward to state 6, and <a title=\"Debugging coroutine handles: The Microsoft Visual C++ compiler, clang, and gcc\" href=\"https:\/\/devblogs.microsoft.com\/oldnewthing\/20211007-00\/?p=105777\"> we know that<\/a> the Microsoft compiler records coroutine states as even numbers starting at 2 (for the initial state), and then increases by two for each suspension point. Therefore, moving to state 6 means suspending for the second time.<\/p>\n<pre>winrt::fire_and_forget DoStuffLater()\r\n{\r\n    co_await winrt::resume_after(100ms);\r\n    co_await winrt::resume_foreground(GetDispatcherQueue());\r\n    DoStuff();\r\n}\r\n<\/pre>\n<p>Okay, good, that second suspension theory lines up with the code: The second suspension is a call to <code>resume_foreground<\/code>, and the code showed that we were calling <code>resume_foreground<\/code>.<\/p>\n<p>And we see the bug: The code is storing the result of <code>await_suspend<\/code> into the coroutine frame. This is something I called out in my <a title=\"C++ coroutines: Getting started with awaitable objects\" href=\"https:\/\/devblogs.microsoft.com\/oldnewthing\/20191209-00\/?p=103195\"> C++ coroutines: Getting started with awaitable objects<\/a> article:<\/p>\n<blockquote class=\"q\"><p>Therefore, it is important that your awaiter not use its <code>this<\/code> pointer once it has arranged for the handle to be invoked somehow, because the <code>this<\/code> pointer may no longer be valid.<\/p><\/blockquote>\n<p>In this case, not only did the awaiter get destructed, the entire coroutine frame was destructed!<\/p>\n<p>The compiler team confirmed that this is a known code-generation bug, <a href=\"https:\/\/devblogs.microsoft.com\/cppblog\/cpp20-coroutine-improvements-in-visual-studio-2019-version-16-11\/\"> fixed in versions 16.11 and 17.0<\/a>.<\/p>\n<p>If you are stuck on 16.10 or older, you will have to work around the problem. From my investigation, it seems that the code generation problem occurs when you have an <code>await_suspend<\/code> that returns <code>bool<\/code>. In C++\/WinRT, there are only four places where this happens:<\/p>\n<ul>\n<li><code>resume_foreground(Windows::System::CoreDispatcher)<\/code><\/li>\n<li><code>resume_foreground(Microsoft::System::CoreDispatcher)<\/code><\/li>\n<li><code>deferrable_event_args.wait_for_deferrals()<\/code><\/li>\n<li><code>final_suspend<\/code><\/li>\n<\/ul>\n<p>In the first two cases, you can work around the problem by switching to <a href=\"https:\/\/github.com\/microsoft\/wil\/blob\/492c01bb535daadf719d4445d6107aadf1e60812\/include\/wil\/cppwinrt.h#L481\"> the <code>wil::resume_foreground<\/code> function<\/a>, which addresses this and other design issues with the original <code>winrt::resume_foreground<\/code> function.<\/p>\n<p>If you&#8217;d rather not pull in another library, and you don&#8217;t want to upgrade your compiler, you can work around the problem by using an explicit continuation-passing model:<\/p>\n<pre>winrt::fire_and_forget DoStuffLater()\r\n{\r\n    co_await winrt::resume_after(100ms);\r\n    GetDispatcherQueue().EnqueueAsync([=]()\r\n    {\r\n        DoStuffLater();\r\n    });\r\n}\r\n<\/pre>\n<p>In the last case (<code>final_<wbr \/>suspend<\/code>), my exploration suggests that the code generation problem does not occur in that case, so we&#8217;re okay there.<\/p>\n<p>But upgrade your compiler if you can.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Applying what we know about coroutines.<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-107233","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"<p>Applying what we know about coroutines.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/107233","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=107233"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/107233\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=107233"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=107233"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=107233"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}