{"id":98665,"date":"2018-05-03T07:00:00","date_gmt":"2018-05-03T21:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/?p=98665"},"modified":"2019-03-13T00:43:28","modified_gmt":"2019-03-13T07:43:28","slug":"20180503-00","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20180503-00\/?p=98665","title":{"rendered":"Avoiding deadlocks when cancelling a thread pool callback, part 1: External callback data"},"content":{"rendered":"

We saw last time how to use the thread pool cleanup functions to manage the lifetime of the context data. But if the callback function tries to call into the thread that is calling Wait­For­Threadpool­Timer­Callbacks<\/code>, then you have a deadlock. The callback cannot proceed until Wait­For­Threadpool­Timer­Callbacks<\/code> returns, but Wait­For­Threadpool­Timer­Callbacks<\/code> won’t return until the callback completes. <\/p>\n

You can find yourself in this situation without realizing it. The callback function might send a message to a window that is owned by the waiting thread. Or it could invokes a method on an apartment-threaded object that belongs to the waiting thread. Or it could attempt to enter a critical section that is held by the waiting thread.¹ In many cases, the waiting thread is cleaning up an object in its destructor, so you don’t even control the locks that may be held at that point. <\/p>\n

This is where Disassociate­Current­Thread­From­Callback<\/code><\/a> enters the picture. The Disassociate­Current­Thread­From­Callback<\/code> function tells the thread pool, “For the purpose of waiting until all callbacks are complete, consider this callback to be complete even though it’s still running.” You can think of this as the Reply­Message<\/code> of the thread pool. This means that functions like Wait­For­Threadpool­Xxx­Callbacks<\/code> will return, but other functions like Xxx­When­Callback­Returns<\/code> won’t be fooled. They will wait for the callback to return for real before setting the event or leaving the critical section or whatever. <\/p>\n

Let’s figure out how to take advantage of this. <\/p>\n

We make the context data for the callback function a thread-safe reference-counted object. Typical examples are a COM pointer to an agile object, and a reference to a std::shared_ptr<\/code>.² The first thing the callback function does is to increment the reference count on the object and save it in an RAII object. For a COM pointer, it would be creating a COM smart pointer around it (say, converting it to a WRL::ComPtr<\/code>). For a std::shared_ptr<\/code> it would be copying the std::shared_ptr<\/code> to a local variable. <\/p>\n

Once the context data has been safely referenced, you call Disassociate­Current­Thread­From­Callback<\/code> to release the waiting thread, if any. <\/p>\n

At this point, you can do your work with the captured strong reference (the WRL::ComPtr<\/code> or the local copy of the std::shared_ptr<\/code>) and stop using the inbound context parameter, because it is no longer valid; the waiting thread may have destroyed it. <\/p>\n

When your callback function completes, the RAII type will release the reference to the context data, and if that was the last reference, it will destroy the context data. <\/p>\n

We start with this helper class.<\/p>\n

\nstruct TpTimerDeleter\n{\n void operator()(PTP_TIMER timer)\n {\n  SetThreadpoolTimer(timer, nullptr, 0, 0);\n  WaitForThreadpoolTimerCallbacks(timer, true);\n  CloseThreadpoolTimer(timer);\n }\n};\n<\/pre>\n
This deleter class performs the standard synchronous shutdown of a thread pool timer, as noted in the documentation<\/a>. <\/p>\n
Here’s how we use it to build a thread pool callback that doesn’t deadlock at cancellation. We’ll start with one based on WRL::ComPtr<\/code>: <\/p>\n
\nclass ObjectWithTimer\n{\npublic:\n StartTimer();\n StopTimer();\n\nprivate:\n static void CALLBACK TimerCallback(\n  PTP_CALLBACK_INSTANCE instance,\n  void* context, PTP_TIMER timer);\n\n WRL::ComPtr<AgileContextData> contextData;<\/font> \/\/ ComPtr-specific code\n std::unique_ptr<TP_TIMER, TpTimerDeleter> timer;\n};\n<\/pre>\n
Having a std::unique_ptr<\/code> automatically makes the class non-copyable. We’ll see soon why it’s okay to let the object be movable. <\/p>\n
Exercise<\/b>: Why did I declare the timer<\/code> member after the contextData<\/code> member? (Normally, I don’t answer the exercises in the main body of the article, but this is an important detail, so I’ll answer it at the end.) <\/p>\n
\nvoid ObjectWithTimer::StartTimer()\n{\n \/\/ Error checking elided for expository purposes.\n timer = CreateThreadpoolTimer(\n    TimerCallback,\n    contextData.Get(),<\/font> \/\/ ComPtr-specific code\n    nullptr);\n\n SetThreadpoolTimer(timer, ...);\n}\n<\/pre>\n
The StartTimer<\/code> method assumes that the contextData<\/code> method has been initialized (presumably by methods not shown) and that the timer has not already been started. It creates the timer with the raw COM pointer as the reference data. This is a non-refcounted pointer, so we have to make sure it remains valid for as long as the callback is potentially-callable. Once we create the timer, we start it by calling Set­Threadpool­Timer<\/code> and passing the timer parameters (not shown here). <\/p>\n
The context parameter passed to the callback is the thing we have to worry about if the Object­With­Timer<\/code> gets moved. In this case, it’s okay to move the Object­With­Timer<\/code> because the context parameter doesn’t point to the Object­With­Timer<\/code>; it is a raw COM pointer to an object that won’t move. Therefore, the Object­With­Timer<\/code> is movable. <\/p>\n
\nvoid ObjectWithTimer::TimerCallback(PTP_CALLBACK_INSTANCE instance,\n  void* context, PTP_TIMER timer)\n{\n \/\/ ComPtr-specific code\n WRL::ComPtr<AgileContextData> contextData{\n    reinterpret_cast<AgileContextData*>(context) };<\/font>\n\n context = nullptr;\n\n DisassociateCurrentThreadFromCallback(instance);\n\n ... do stuff with contextData ...\n}\n<\/pre>\n
The callback function starts by taking our weak pointer and converting it to a strong pointer. We designed our code so that the raw COM pointer is valid for as long as the callback is potentially-callable, so we know that it is valid here. We put it inside a WRL::ComPtr<\/code> to make it a strong reference. <\/p>\n
And then here’s the interesting part: We call Disassociate­Current­Thread­From­Callback<\/code><\/a> to tell the thread pool to release anybody who is waiting for the callback to complete. The callback is still running<\/i>, but anybody who is waiting for it to complete is free to proceed anyway. We can do this because we have captured the information from the context<\/code> parameter, and the main object can free it. (To ensure we don’t access the context<\/code> parameter by mistake, we also set context<\/code> to nullptr<\/code>.) <\/p>\n
We then perform our callback operation with the strong reference in contextData<\/code>. This can call back into the main thread because the main thread is no longer stuck in Wait­For­Threadpool­Timer­Callbacks<\/code>, It will eventually return to its event dispatch loop, or release its locks, or whatever it is that needs to happen for the thread pool thread to be able to communicate with the main thread. <\/p>\n
The other half of the dance comes when we want to stop the timer. <\/p>\n
\nvoid ObjectWithTimer::StopTimer()\n{\n timer.reset();\n contextData.Reset();<\/font> \/\/ ComPtr-specific code\n}\n<\/pre>\n
To stop the timer, we let the TpTimerDeleter<\/code> do the heavy lifting of checking if we have a timer and if so, shutting it down cleanly. <\/p>\n
Once that’s done, we can safely release our reference to the context data. If the callback is running, it will have its own reference. <\/p>\n
For simplicity, this code doesn’t try to save the thread pool timer object for future use. One of the features of the thread pool functions is that object creation preallocates all resources. Once you’ve created the thread pool timer successfully, all other operations will always succeed (assuming they are used correctly, of course). Therefore, what we could’ve done is allocate the PTP_TIMER<\/code> at construction (throwing if not possible), and the have the Start­Timer<\/code> and Stop­Timer<\/code> methods merely reconfigure the timer and (in the case of Stop­Timer<\/code>) wait for the callbacks to drain. <\/p>\n
Note that in order to avoid the deadlock, we have to accept that the callback may run after the timer has been stopped<\/i>. When you have a deadlock, something has to give, and we choose to break the deadlock by letting the callback complete asynchronously. If you need to know when the callback is definitely finished, you could add an event that gets signaled when the COM object is destructed, so that the caller knows when everything is finally finished. <\/p>\n
Adapting the above code to std::shared_ptr<\/code> is not too difficult. The tricky part is that we cannot pass a raw pointer to the context data as our reference data, because raw C++ objects are not reference-counted. Instead, we pass a pointer to the std::shared_ptr<\/code>, which is the thing with the reference count. <\/p>\n
\nclass ObjectWithTimer\n{\npublic:\n \/\/ Make object non-movable\n ObjectWithTimer(ObjectWithTimer&&) = delete;\n ObjectWithTimer operator=(ObjectWithTimer&&) = delete;<\/font>\n\n StartTimer();\n StopTimer();\n\nprivate:\n static void CALLBACK TimerCallback(\n  PTP_CALLBACK_INSTANCE instance,\n  void* context, PTP_TIMER timer);\n\n std::shared_ptr<AgileContextData> contextData;<\/font> \/\/ shared_ptr-specific code\n std::unique_ptr<TP_TIMER, TpTimerDeleter> timer;\n};\n\nvoid ObjectWithTimer::StartTimer()\n{\n  \/\/ Error checking elided for expository purposes.\n  timer = CreateThreadpoolTimer(\n    TimerCallback,\n    std::addressof(contextData),<\/font> \/\/ shared_ptr-specific code\n    nullptr);\n\n  \/\/ Start the timer\n  SetThreadpoolTimer(timer, ...);\n}\n\nvoid ObjectWithTimer::TimerCallback(PTP_CALLBACK_INSTANCE instance,\n  void* context, PTP_TIMER timer)\n{\n \/\/ Capture the context with a strong reference\n \/\/ shared_ptr-specific code\n std::shared_ptr<AgileContextData> contextData{\n    *reinterpret_cast<\n      std::shared_ptr<AgileContextData>*>(context) };<\/font>\n\n context = nullptr;\n\n DisassociateCurrentThreadFromCallback(instance);\n\n ... do stuff with contextData ...\n}\n\nvoid ObjectWithTimer::StopTimer()\n{\n  timer.reset();\n  contextData.reset();<\/font> \/\/ shared_ptr-specific code\n}\n<\/pre>\n
The principle is the same here as before, but the implementation is made more complicated by the fact that the std::shared_ptr<\/code> is not necessarily (and in fact usually isn’t) the size of a pointer, so it doesn’t fit in the context pointer. We have to pass a pointer to the std::shared_ptr<\/code>. <\/p>\n
 Exercise<\/b>: We could have written &contextData<\/code> instead of std::addressof(contextData)<\/code>. Why did I use std::addressof(contextData)<\/code>? <\/p>\n
The fact that a std::shared_ptr<\/code> doesn’t fit in a pointer means that the std::shared_ptr<\/code> cannot move, because it is still being referenced by the context pointer passed to the thread pool callback. We can make the Object­With­Timer<\/code> object movable by putting the std::shared_ptr<\/code> inside a std::unique_ptr<\/code>, and passing the raw pointer to the std::shared_ptr<\/code>. The std::unique_ptr<\/code> will transfer the pointer to the moved-to object, and the std::shared_ptr<\/code> itself doesn’t move. <\/p>\n
\nclass ObjectWithTimer\n{\npublic:\n \/\/ Make object movable again\n \/\/ ObjectWithTimer(ObjectWithTimer&&) = delete;<\/strike>\n \/\/ ObjectWithTimer operator=(ObjectWithTimer&&) = delete;<\/strike><\/font>\n\n StartTimer();\n StopTimer();\n\nprivate:\n static void CALLBACK TimerCallback(\n  PTP_CALLBACK_INSTANCE instance,\n  void* context, PTP_TIMER timer);\n\n std::unique_ptr<\n   shared_ptr<AgileContextData>> contextData;<\/font> \/\/ unique_ptr-specific code\n std::unique_ptr<TP_TIMER, TpTimerDeleter> timer;\n};\n\nvoid ObjectWithTimer::StartTimer()\n{\n  \/\/ Error checking elided for expository purposes.\n  timer = CreateThreadpoolTimer(\n    TimerCallback,\n    contextData.get(),<\/font> \/\/ unique_ptr-specific code\n    nullptr);\n\n  \/\/ Start the timer\n  SetThreadpoolTimer(timer, ...);\n}\n\nvoid ObjectWithTimer::TimerCallback(PTP_CALLBACK_INSTANCE instance,\n  void* context, PTP_TIMER timer)\n{\n \/\/ Capture the context with a strong reference\n \/\/ no change here\n std::shared_ptr<AgileContextData> contextData{\n    *reinterpret_cast<\n      std::shared_ptr<AgileContextData>*>(context) };<\/font>\n\n context = nullptr;\n\n DisassociateCurrentThreadFromCallback(instance);\n\n ... do stuff with contextData ...\n}\n\nvoid ObjectWithTimer::StopTimer()\n{\n  timer.reset();\n  contextData.reset();<\/font> \/\/ no change here\n}\n<\/pre>\n
The idea in all the cases is that we keep a reference-counted object in the contextData<\/code> and provide the callback a way to convert its context parameter to a strong reference. Since we are careful always to keep the context parameter valid as long as the callback is potentially-callable, this conversion is straightforward: Just create your own strong reference from the raw pointer. <\/p>\n
Often, the context for the callback is the containing object itself, rather than some external data. We’ll explore that scenario next time, because this article is too long as it is. <\/p>\n
¹ The critical section case is easy to imagine: You might have a critical section that protects access to object state. The waiting thread owns the critical section because it’s trying to clean up the object’s thread pool timer. The callback is trying to acquire the critical section because it wants to access the object’s state as part of the callback operation. <\/p>\n
² Be careful with the std::shared_ptr<\/code><\/a>. Copying a std::shared_ptr<\/code> is thread-safe, but mutating it is not, so you should initialize your std::shared_ptr<\/code> with the context structure and not modify the std::shared_ptr<\/code> until you are sure that no other threads are accessing it. <\/p>\n","protected":false},"excerpt":{"rendered":"
Breaking the deadlock by disassociating from the thread pool.<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-98665","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"
Breaking the deadlock by disassociating from the thread pool.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/98665","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=98665"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/98665\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=98665"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=98665"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=98665"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}