{"id":11613,"date":"2011-02-02T07:00:00","date_gmt":"2011-02-02T07:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/2011\/02\/02\/ready-cancel-wait-for-it-part-1\/"},"modified":"2011-02-02T07:00:00","modified_gmt":"2011-02-02T07:00:00","slug":"ready-cancel-wait-for-it-part-1","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20110202-00\/?p=11613","title":{"rendered":"Ready… cancel… wait for it! (part 1)"},"content":{"rendered":"

\nOne of the cardinal rules of the OVERLAPPED<\/code>\nstructure is the OVERLAPPED<\/code> structure\nmust remain valid until the I\/O completes<\/i>.\nThe reason is that the OVERLAPPED<\/code> structure\nis\n\nmanipulated by address rather than by value<\/a>.\n<\/p>\n

\nThe word complete<\/i> here has a specific technical meaning.\nIt doesn’t mean “must remain valid until you are no longer interested\nin the result of the I\/O.”\nIt means that the structure must remain valid until\nthe I\/O subsystem has signaled that the I\/O operation\nis finally over, that there is nothing left to do,\nit has passed on:\nYou have an ex-I\/O operation.\n<\/p>\n

\nNote that\nan I\/O operation can complete successfully, or it can\ncomplete unsuccessfully.\nCompletion is not the same as success.\n<\/p>\n

\nA common mistake when performing overlapped I\/O\nis issuing a cancel and immediately freeing the OVERLAPPED<\/code>\nstructure.\nFor example:\n<\/p>\n

\n\/\/ this code is wrong\n HANDLE h = ...; \/\/ handle to file opened as FILE_FLAG_OVERLAPPED\n OVERLAPPED o;\n BYTE buffer[1024];\n InitializeOverlapped(&o); \/\/ creates the event etc\n if (ReadFile(h, buffer, sizeof(buffer), NULL, &o) ||\n     GetLastError() == ERROR_IO_PENDING) {\n  if (WaitForSingleObject(o.hEvent, 1000) != WAIT_OBJECT_0) {\n   \/\/ took longer than 1 second - cancel it and give up\n   CancelIo(h);\n   return WAIT_TIMEOUT;\n  }\n  ... use the results ...\n }\n ...<\/i>\n<\/pre>\n
\nThe bug here is that after calling Cancel­Io<\/code>,\nthe function returns without waiting for the Read­File<\/code>\nto complete.\nReturning from the function\nimplicitly frees the automatic variable o<\/code>.\nWhen the Read­File<\/code> finally completes, the I\/O system\nis now writing to stack memory that has been freed and is probably\nbeing reused by another function.\nThe result is impossible to debug:\nFirst of all, it’s a race condition between your code and the I\/O\nsubsystem, and breaking into the debugger doesn’t stop the\nI\/O subsystem<\/i>.\nIf you step through the code, you don’t see the corruption,\nbecause the I\/O completes while you’re broken into the debugger<\/i>.\n<\/p>\n
\nHere’s what happens when the program is run outside the debugger:\n<\/p>\n\n\n\n\n\n\n\n\n
ReadFile<\/td>\n→<\/td>\nI\/O begins<\/td>\n<\/tr>\n
WaitForSingleObject<\/td>\n<\/td>\nI\/O still in progress<\/td>\n<\/tr>\n
WaitForSingleObject times out<\/td>\n<\/tr>\n
CancelIo<\/td>\n→<\/td>\nI\/O cancellation submitted to device driver<\/td>\n<\/tr>\n
return<\/td>\n<\/tr>\n
<\/td>\n<\/td>\nDevice driver was busy reading from the hard drive
\n        Device driver receives the cancellation
\n        Device driver abandons the rest of the read operation
\n        Device driver reports that I\/O has been canceled
\n        I\/O subsystem writes STATUS_CANCELED<\/code>\n        to OVERLAPPED<\/code> structure
\n        I\/O subsystem queues the completion function (if applicable)
\n        I\/O subsystem signals the completion event (if applicable)
\n        I\/O operation is now complete<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
\nWhen the I\/O subsystem receives word from the device driver\nthat the cancellation has completed,\nit performs the usual operations when an I\/O operation completes:\nIt updates the OVERLAPPED<\/code> structure with the results\nof the I\/O operation, and notifies whoever wanted to be notified\nthat the I\/O is finished.\n<\/p>\n
\nNotice that when it updates the OVERLAPPED<\/code> structure,\nit’s updating memory that has already been freed back to the stack,\nwhich means that it’s corrupting the stack of whatever function\nhappens to be running right now.\n(It’s even worse if you happened to catch it while it was in the\nprocess of updating the buffer<\/code>!)\nSince the precise timing of I\/O is unpredictable,\nthe program crashes with memory corruption that keeps changing\neach time it happens.\n<\/p>\n
\nIf you try to debug the program, you get this:\n<\/p>\n\n\n\n\n\n\n\n\n\n\n\n
ReadFile<\/td>\n→<\/td>\nI\/O begins<\/td>\n<\/tr>\n
WaitForSingleObject<\/td>\n<\/td>\nI\/O still in progress<\/td>\n<\/tr>\n
WaitForSingleObject times out<\/td>\n<\/tr>\n
Breakpoint hit on Cancel­Io<\/code> statement
\n        Stops in debugger<\/td>\n<\/tr>\n
Hit F10 to step over the CancelIo call<\/td>\n→<\/td>\nI\/O cancellation submitted to device driver<\/td>\n<\/tr>\n
Breakpoint hit on return<\/code> statement
\n        Stops in debugger<\/td>\n<\/tr>\n
<\/td>\n<\/td>\nDevice driver was busy reading from the hard drive
\n        Device driver receives the cancellation
\n        Device driver abandons the rest of the read operation
\n        Device driver reports that I\/O has been canceled
\n        I\/O subsystem writes STATUS_CANCELED<\/code>\n        to OVERLAPPED<\/code> structure
\n        I\/O subsystem queues the completion function (if applicable)
\n        I\/O subsystem signals the completion event (if applicable)
\n        I\/O operation is now complete<\/td>\n<\/tr>\n
Look at the OVERLAPPED<\/code> structure in the debugger
\n        It says STATUS_CANCELED<\/code><\/td>\n<\/tr>\n
Hit F5 to resume execution
\n        No memory corruption<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
\nBreaking into the debugger changed the timing of the I\/O operation\nrelative to program execution.\nNow, the I\/O completes before the function returns,\nand consequently there is no memory corruption.\nYou look at the OVERLAPPED<\/code> structure and say,\n“See? Immediately on return from the Cancel­Io<\/code> function,\nthe OVERLAPPED<\/code> structure has been updated with the result,\nand the buffer<\/code> contents are not being written to.\nIt’s safe to free them both now.\nTherefore, this can’t be the source of my memory corruption bug.”\n<\/p>\n
\nExcept, of course, that it is.\n<\/p>\n
\nThis is even more crazily insidious because the OVERLAPPED<\/code>\nstructure and the buffer<\/code> are\nupdated by the I\/O subsystem, which means that it happens\nfrom kernel mode<\/i>.\nThis means that\n\nwrite breakpoints set by your debugger won’t fire<\/a>.\nEven if you manage to narrow down the corruption to\n“it happens somewhere in this function”,\nyour breakpoints will never see it as it happens.\nYou’re going to see that the value was good,\nthen a little while later, the value was bad,\nand yet your write breakpoint never fired.\nYou’re then going to declare that the world has gone mad\nand seriously consider a different line of work.\n<\/p>\n
\nTo fix this race condition,\nyou have to delay freeing the OVERLAPPED<\/code> structure\nand the associated buffer<\/code>\nuntil the I\/O is complete and anything else that’s using them\nhas also given up their claim to it.\n<\/p>\n
\n   \/\/ took longer than 1 second - cancel it and give up\n   CancelIo(h);\n   WaitForSingleObject(o.hEvent, INFINITE); \/\/ added\n   \/\/ Alternatively: GetOverlappedResult(h, &o, TRUE);<\/font>\n   return WAIT_TIMEOUT;\n<\/pre>\n
\nThe Wait­For­Single­Object<\/code> after the\nCancel­Io<\/code>\nwaits for the I\/O to complete\nbefore finally returning (and implicitly freeing the OVERLAPPED<\/code>\nstructure and the buffer<\/code> on the stack).\nBetter would be to use\nGetOverlapped­Result<\/code>\nwith bWait = TRUE<\/code>,\nbecause that also handles the case where the hEvent<\/code>\nmember of the OVERLAPPED<\/code> structure is NULL<\/code>.\n<\/p>\n
\nExercise<\/b>:\nIf you retrieve the completion status\nafter canceling the I\/O\n(either by looking at the OVERLAPPED<\/code> structure\ndirectly or by using GetOverlapped­Result<\/code>)\nthere’s a chance that the overlapped result\nwill be something other than STATUS_CANCELED<\/code>\n(or ERROR_CANCELLED<\/code> if you prefer Win32 error codes).\nExplain.\n<\/p>\n
\nExercise<\/b>:\nIf this example had used Read­File­Ex<\/code>,\nthe proposed fix would be incomplete.\nExplain and provide a fix.\nAnswer to come next time, and then we’ll look at another\nversion of this same principle.<\/p>\n","protected":false},"excerpt":{"rendered":"
One of the cardinal rules of the OVERLAPPED structure is the OVERLAPPED structure must remain valid until the I\/O completes. The reason is that the OVERLAPPED structure is manipulated by address rather than by value. The word complete here has a specific technical meaning. It doesn’t mean “must remain valid until you are no longer […]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-11613","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"
One of the cardinal rules of the OVERLAPPED structure is the OVERLAPPED structure must remain valid until the I\/O completes. The reason is that the OVERLAPPED structure is manipulated by address rather than by value. The word complete here has a specific technical meaning. It doesn’t mean “must remain valid until you are no longer […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/11613","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=11613"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/11613\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=11613"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=11613"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=11613"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}