{"id":105028,"date":"2021-03-31T07:00:00","date_gmt":"2021-03-31T14:00:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/oldnewthing\/?p=105028"},"modified":"2021-04-01T08:17:01","modified_gmt":"2021-04-01T15:17:01","slug":"20210331-00","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20210331-00\/?p=105028\/","title":{"rendered":"C++ coroutines: The initial and final suspend, and improving our return_value method"},"content":{"rendered":"

Last time, we had a basic implementation of a promise type<\/a> but left with the question of what the initial_suspend<\/code> and final_suspend<\/code> are for.<\/p>\n

When the compiler encounters a coroutine function, it performs multiple rewrite passes. One of the early passes produces the following:<\/p>\n

return_type MyCoroutine(args...)\r\n{\r\n    create coroutine state<\/i>\r\n    copy parameters to coroutine frame<\/i>\r\n    promise_type p;\r\n    auto return_object = p.get_return_object();\r\n\r\n    try {\r\n        co_await p.initial_suspend(); \/\/ \u00b9\r\n        coroutine function body\u00b2<\/i>\r\n    } catch (...) {\r\n        p.unhandled_exception();\r\n    }\r\n    co_await p.final_suspend();\r\n    destruct promise p<\/i>\r\n    destruct parameters in coroutine frame<\/i>\r\n    destroy coroutine state<\/i>\r\n}\r\n<\/pre>\n
This rewrite is where the initial_suspend<\/code> and final_suspend<\/code> enter the picture.<\/p>\n
After constructing the promise p<\/code>, it calls p.get_return_object()<\/code> to obtain the object that is passed back to the caller. Next, the coroutine awaits whatever initial_suspend()<\/code> returns.<\/p>\n
There are two common choices for initial_suspend()<\/code>: If you return a suspend_never<\/code> or some other awaiter that doesn’t suspend the coroutine, then the coroutine keeps running until the first suspending co_await<\/code>. This is the model for “hot-start” coroutines which execute synchronously during their construction and don’t return an object until the first suspension inside the function body.<\/p>\n
On the other hand, if you return an awaiter which results in the coroutine suspending, then the caller gets its return object right away. Nothing in the coroutine function body has executed yet. When the caller eventually performs a co_await<\/code> on the return object, the return object resumes the coroutine.\u00b3 This is the model for “cold-start” coroutines which don’t begin executing until they are awaited.<\/p>\n
Conversely, there is a final_suspend<\/code> at the end of the coroutine after the coroutine function body has finished. This gives you a chance to do any extra cleanup, as well as perform some other magic we’ll look at in a future installment.<\/p>\n
One way to improve in our implementation of the coroutine_traits<result_holder><\/code> promise type is our handling of return_value<\/code>. First, there’s a simple improvement of accepting the value by reference rather than value, so we can forward it into the result_holder<\/code>, thereby avoiding some extra move operations.<\/p>\n
            void return_value(T&& v) const\r\n            {\r\n                holder.set_result(std::move(v));\r\n            }<\/span>\r\n\r\n            void return_value(T const& v) const\r\n            {\r\n                holder.set_result(v);\r\n            }\r\n<\/pre>\n
But there’s a bigger problem to fix: We release the waiting coroutines too soon.\u2074<\/p>\n
Consider a coroutine function that goes like this:<\/p>\n
result_holder<int> SomethingAsync()\r\n{\r\n    auto guard = std::lock_guard{ mutex };\r\n    co_return 42;\r\n}\r\n<\/pre>\n
When this function reaches the co_return<\/code>, the compiler generates a call to return_value<\/code>, and our implementation of return_value<\/code> sets the result into the holder, which immediately releases any waiting coroutines.<\/p>\n
But those waiting coroutines might want to acquire the same mutex that the Something\u00adAsync<\/code> function still owns. You end up in a hurry-up-and-wait situation, where we wake up a coroutine, only to have it block immediately.<\/p>\n
The scenario is even more dire if we resume the waiting coroutines synchronously, because the waiting coroutine may want to acquire the mutex, but it can’t because Something\u00adAsync<\/code> still owns the mutex, and Something\u00adAsync<\/code> won’t resume execution until after return_value<\/code> returns, which can’t happen because it’s waiting for the resumed coroutine to reach its next suspension point.<\/p>\n
More generally, local variables are still alive at the point of the co_return<\/code>, so any resources held by those local variables are still active at the point of the return_value<\/code>.<\/p>\n
The solution is to break the return_value<\/code> into two steps. The first step executes immediately: Saving the value or exception into the holder<\/code>. But we don’t wake up the waiting coroutines yet. Leave them suspended for a little while longer.<\/p>\n
After the return value or exception is captured, the local variables in the coroutine function body are destructed when we leave the scope of the try<\/code> block that wraps the coroutine function body. Outside the try<\/code> block, we perform a final_suspend<\/code>, and that’s where we can take the second step of waking the waiting coroutines.<\/p>\n
To implement this, we need to add a few new methods to our result_holder<\/code>:<\/p>\n
struct result_holder_state\r\n{\r\n    ...\r\n\r\n    void stage_result(T v)\r\n    {\r\n        if (kind.load(std::memory_order_relaxed)\r\n            == result_kind::unset) {\r\n            new (std::addressof(result.wrap))\r\n                wrapper{ std::forward<T>(v) };\r\n            kind.store(result_kind::value,\r\n                std::memory_order_release);\r\n        }\r\n    }\r\n\r\n    void stage_exception(std::exception_ptr ex) noexcept\r\n    {\r\n        if (kind.load(std::memory_order_relaxed)\r\n            == result_kind::unset) {\r\n            new (std::addressof(result.ex))\r\n                std::exception_ptr{ std::move(ex) };\r\n            kind.store(result_kind::exception,\r\n                std::memory_order_release);\r\n        }\r\n    }\r\n\r\n    void complete(node_list& list) noexcept\r\n    {\r\n        this->resume_all(list);\r\n    }\r\n};\r\n<\/pre>\n
We break the set_result<\/code> into two parts: stage_result<\/code> puts the result in the state, but doesn’t resume anybody yet. The resumption of waiting coroutines happens when we call complete<\/code>.<\/p>\n
We hook up these methods to the main result_holder<\/code> class:<\/p>\n
struct result_holder\r\n{\r\n    ...\r\n\r\n    void stage_result(T result) const\r\n    {\r\n        this->get_state().stage_result(std::move(result));\r\n    }\r\n\r\n    void stage_exception(std::exception_ptr ex) const noexcept\r\n    {\r\n        this->get_state().stage_exception(std::move(ex));\r\n    }\r\n\r\n    void complete() const noexcept\r\n    {\r\n        this->action_impl(&state::complete);\r\n    }\r\n};\r\n<\/pre>\n
And we revise our promise to use these new methods:<\/p>\n
            void return_value(T&& v) const\r\n            {\r\n                holder.stage_result(std::move(v));\r\n            }\r\n\r\n            void return_value(T const& v) const\r\n            {\r\n                holder.stage_result(v);\r\n            }\r\n\r\n            suspend_never final_suspend() const noexcept\r\n            {\r\n                holder.complete();\r\n                return{};\r\n            }\r\n<\/pre>\n
Okay, so we’ve fixed the problem of resuming the waiting coroutines too soon. There’s another improvement we can make, but there’s another topic I want to cover first, which I’ll do next time.<\/p>\n
\u00b9 The actual transformation is more complicated than described here, but the simplified version will suffice for now. One discrepancy that is worth noting here is that an exception that occurs when creating the initial suspend object is not<\/i> caught by the try<\/code> statement; instead, it propagates synchronously out of the coroutine. However, an exception that occurs during<\/i> the co_await<\/code> is indeed caught by the try<\/code> statement and ends up captured into the promise.<\/p>\n
\u00b2 If there is a function try around the entire function, the try<\/code> is considered to be part of the function body. See [dcl.fct.def.general]<\/b> for the formal definition of “function body”. This transformation of function try is performed even before the transformation described above.<\/p>\n
\u00b3 I’m assuming of course that you provided the return object a way to resume the coroutine when it is awaited. We’ll explore possible ways of doing this in future installments.<\/p>\n
\u2074 My thanks to Gor Nishanov for pointing out this issue to me.<\/p>\n","protected":false},"excerpt":{"rendered":"
Why would you stop before you start?<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-105028","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"
Why would you stop before you start?<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/105028","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=105028"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/105028\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=105028"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=105028"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=105028"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}