{"id":105078,"date":"2021-04-12T07:00:00","date_gmt":"2021-04-12T14:00:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/oldnewthing\/?p=105078"},"modified":"2023-10-09T10:34:59","modified_gmt":"2023-10-09T17:34:59","slug":"20210412-00","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20210412-00\/?p=105078","title":{"rendered":"C++ coroutines: The lifetime of objects involved in the coroutine function"},"content":{"rendered":"

We finally<\/a> hooked up the last missing piece of our coroutine promise implementation. Before we can look at the tradeoffs we’ve made, let’s step back and follow the lifetime of the various objects involved in the coroutine function.<\/p>\n

Suppose we have a coroutine that goes like this:<\/p>\n

task SomeCoroutine(Parameter p)\r\n{\r\n    Local1 l1;\r\n    co_await l1.Method();\r\n    co_return l1.Result();\r\n}\r\n<\/pre>\n
The coroutine state will look something like this:<\/p>\n
<\/p>\n\n\n\n\n\n\n\n\n
 <\/td>\n <\/td>\nCoroutine state<\/th>\n<\/tr>\n
 <\/td>\n <\/td>\nbookkeeping<\/td>\n<\/tr>\n
promise<\/td>\n\n
\u00a0<\/div>\n<\/td>\n
(you write this)<\/td>\n<\/tr>\n
coroutine frame<\/td>\n\n
\u00a0
\n\u00a0
\n\u00a0
\n\u00a0<\/div>\n<\/td>\n
Parameter p<\/tt><\/td>\n<\/tr>\n
Local l1<\/tt><\/td>\n<\/tr>\n
temporaries<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
The coroutine state consists of three parts: Compiler-defined bookkeeping to keep track of the progress of the coroutine, the promise object (which you define), and the frame (which contains the parameters, locals, and any temporaries).<\/p>\n
We’ve written two kinds of promises, and they look like this:<\/p>\n
<\/p>\n\n\n\n\n\n\n\n\n\n
Coroutine state<\/th>\n <\/td>\n <\/td>\n\u00a0<\/td>\nCoroutine state<\/th>\n<\/tr>\n
bookkeeping<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nbookkeeping<\/td>\n<\/tr>\n
shared_ptr<\/tt><\/td>\n\u2192<\/td>\nReturnValue<\/tt><\/td>\n\u00a0<\/td>\nReturnValue<\/tt><\/td>\n<\/tr>\n
Parameter p<\/tt><\/td>\n <\/td>\nother stuff<\/td>\n\u00a0<\/td>\nother stuff<\/td>\n<\/tr>\n
Local l1<\/tt><\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nParameter p<\/tt><\/td>\n<\/tr>\n
temporaries<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nLocal l1<\/tt><\/td>\n<\/tr>\n
 <\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\ntemporaries<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
The first kind is the simple one, where the promise object is a shared_ptr<\/code> to a result_holder<\/code> object, and the result_holder<\/code> holds the return value and other internal bookkeeping. In the first kind of promise, there are two allocations, one for the coroutine state, and one for the result_holder<\/code>.<\/p>\n
The second kind is the one where we inlined the result_holder<\/code> object into the promise. In the second kind of promise, there is just one allocation, namely for the coroutine state. The promises consists of the result_holder<\/code> itself, so there is no second allocation.<\/p>\n
Let’s track the memory usage and object lifetimes.<\/p>\n
When you call the coroutine, the first thing that happens it that the compiler allocates a coroutine state and sets up some bookkeeping. In the diagrams, boxes with heavy solid lines contain live data, whereas boxes with thin lines represent uninitialized memory. Solid lines delimit allocated memory, and dotted lines go around memory that has not yet been allocated.<\/p>\n
<\/p>\n\n\n\n\n\n\n\n\n\n
Coroutine state<\/th>\n <\/td>\n <\/td>\n\u00a0<\/td>\nCoroutine state<\/th>\n<\/tr>\n
bookkeeping<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nbookkeeping<\/td>\n<\/tr>\n
shared_ptr<\/tt><\/td>\n\u290d<\/td>\nReturnValue<\/tt><\/td>\n\u00a0<\/td>\nReturnValue<\/tt><\/td>\n<\/tr>\n
Parameter p<\/tt><\/td>\n <\/td>\nother stuff<\/td>\n\u00a0<\/td>\nother stuff<\/td>\n<\/tr>\n
Local l1<\/tt><\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nParameter p<\/tt><\/td>\n<\/tr>\n
temporaries<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nLocal l1<\/tt><\/td>\n<\/tr>\n
 <\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\ntemporaries<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
At the start of the coroutine, the compiler allocates the coroutine state, but only the compiler bookkeeping portion is live. The promise and the frame remain still uninitialized.<\/p>\n
Next, the parameters are moved into the coroutine frame. If a parameter is a reference, then the reference is moved into the frame as a reference. In our example, this means that Parameter<\/code> is constructed.<\/p>\n
<\/p>\n\n\n\n\n\n\n\n\n\n
Coroutine state<\/th>\n <\/td>\n <\/td>\n\u00a0<\/td>\nCoroutine state<\/th>\n<\/tr>\n
bookkeeping<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nbookkeeping<\/td>\n<\/tr>\n
shared_ptr<\/tt><\/td>\n\u290d<\/td>\nReturnValue<\/tt><\/td>\n\u00a0<\/td>\nReturnValue<\/tt><\/td>\n<\/tr>\n
Parameter p<\/tt><\/td>\n <\/td>\nother stuff<\/td>\n\u00a0<\/td>\nother stuff<\/td>\n<\/tr>\n
Local l1<\/tt><\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nParameter p<\/tt><\/td>\n<\/tr>\n
temporaries<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nLocal l1<\/tt><\/td>\n<\/tr>\n
 <\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\ntemporaries<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
At this point, the coroutine bookkeeping and the Parameter<\/code> in the coroutine state have been initialized.<\/p>\n
Next, the compiler constructs the promise object. In our first example, the constructor for the promise object also constructs the shared_ptr<\/code>, which creates the promise state object that holds space for the Return\u00adValue<\/code> as well as other promise bookkeeping. In our second example, the promise state is inlined into the promise. Again, memory is reserved for the Return\u00adValue<\/code>, but nothing is in it yet.<\/p>\n
This is what we have so far:<\/p>\n
<\/p>\n\n\n\n\n\n\n\n\n\n
Coroutine state<\/th>\n <\/td>\n <\/td>\n\u00a0<\/td>\nCoroutine state<\/th>\n<\/tr>\n
bookkeeping<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nbookkeeping<\/td>\n<\/tr>\n
shared_ptr<\/code><\/td>\n\u2192<\/td>\nReturnValue<\/code><\/td>\n\u00a0<\/td>\nReturnValue<\/code><\/td>\n<\/tr>\n
Parameter p<\/tt><\/td>\n <\/td>\nother stuff<\/td>\n\u00a0<\/td>\nother stuff<\/td>\n<\/tr>\n
Local l1<\/code><\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nParameter p<\/tt><\/td>\n<\/tr>\n
temporaries<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nLocal l1<\/code><\/td>\n<\/tr>\n
 <\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\ntemporaries<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
In the first example, the shared_ptr<\/code> for the promise object has been initialized, and it points to a newly-allocated result_holder<\/code>. In that result_holder<\/code>, the bookkeeping is initialized, but the return value portion remains uninitialized. The return value won’t get initialized until the coroutine exits either by reaching a co_return<\/code> or by exiting with an exception.<\/p>\n
In the second example, the promise object is itself just a result_holder<\/code>; there is no shared_ptr<\/code> and no second allocation. Again, the bookkeeping for the result_holder<\/code> has been initialized, but the portion that holds the return value won’t get initialized until there is something to return.<\/p>\n
As the coroutine runs, local variables and temporaries are constructed and destructed according to the usual rules of the C++ language. Those local variables and temporaries appear and disappear in the frame. In our example, this means that Local<\/code> gets constructed in the frame, and then a temporary is created to hold the return value from l1.Method()<\/code>. We then go through the standard co_await<\/code> machinery: From that temporary value, we obtain an awaiter, which is another temporary value, which also goes into the frame. The awaiter arranges for the coroutine to resume, and at the resumption, the temporaries are destructed.<\/p>\n
Just before the co_return<\/code>, we have this:<\/p>\n
<\/p>\n\n\n\n\n\n\n\n\n\n
Coroutine state<\/th>\n <\/td>\n <\/td>\n\u00a0<\/td>\nCoroutine state<\/th>\n<\/tr>\n
bookkeeping<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nbookkeeping<\/td>\n<\/tr>\n
shared_ptr<\/tt><\/td>\n\u2192<\/td>\nReturnValue<\/tt><\/td>\n\u00a0<\/td>\nReturnValue<\/tt><\/td>\n<\/tr>\n
Parameter p<\/tt><\/td>\n <\/td>\nother stuff<\/td>\n\u00a0<\/td>\nother stuff<\/td>\n<\/tr>\n
Local l1<\/tt><\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nParameter p<\/tt><\/td>\n<\/tr>\n
temporaries<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nLocal l1<\/tt><\/td>\n<\/tr>\n
 <\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\ntemporaries<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
In both versions of the coroutine state, the compiler bookkeeping remains initialized, as well as the bookkeeping for the result_holder<\/code>, the captured parameter, and the local variable. In the first example, we also have a shared_ptr<\/code> to the result_holder<\/code>, but in the second example, result_holder<\/code> is embedded in the coroutine state, so no pointer or second allocation is required. In the frame, the parameter remains initialized as well as the local. The temporary is now uninitialized because the lifetime of the temporary has ended.<\/p>\n
We now reach the co_return<\/code>. The co_return<\/code>‘d value is Let’s say that the Result()<\/code> method returns a Return\u00adValue<\/code> object by value. That rvalue reference to the Return\u00adValue<\/code> is passed to the promise’s return_value<\/code> method. In our first example, the promise move-constructs the Return\u00adValue<\/code> in the external promise state. In our second example, the promise move-constructs it in the embedded promise state.<\/p>\n
<\/p>\n\n\n\n\n\n\n\n\n\n
Coroutine state<\/th>\n <\/td>\n <\/td>\n\u00a0<\/td>\nCoroutine state<\/th>\n<\/tr>\n
bookkeeping<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nbookkeeping<\/td>\n<\/tr>\n
shared_ptr<\/tt><\/td>\n\u2192<\/td>\nReturnValue<\/tt><\/td>\n\u00a0<\/td>\nReturnValue<\/tt><\/td>\n<\/tr>\n
Parameter p<\/tt><\/td>\n <\/td>\nother stuff<\/td>\n\u00a0<\/td>\nother stuff<\/td>\n<\/tr>\n
Local l1<\/tt><\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nParameter p<\/tt><\/td>\n<\/tr>\n
temporaries<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nLocal l1<\/tt><\/td>\n<\/tr>\n
 <\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\ntemporaries<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
At this point, the Return\u00adValue<\/code> object in the result_holder<\/code> is initialized. In the first example, the result_holder<\/code> is external to the coroutine state; in the second example, it is embedded inside the coroutine state.<\/p>\n
At the end of the co_return<\/code> statement, the temporary Return\u00adValue<\/code> is destructed. This Return\u00adValue<\/code> does not have a lifetime that extends across a suspension point, so it doesn’t have to go into the coroutine frame. It can just go on the regular stack.<\/p>\n
The coroutine now completes, and any remaining local variables are destructed. In this case, it means that the Local<\/code> in the coroutine frame is now destructed. The coroutine frame now consists only of the compiler bookkeeping, the promise, and the parameters. All of the locals and temporaries have been destructed. The memory for them is just sitting around unused.<\/p>\n
<\/p>\n\n\n\n\n\n\n\n\n\n
Coroutine state<\/th>\n <\/td>\n <\/td>\n\u00a0<\/td>\nCoroutine state<\/th>\n<\/tr>\n
bookkeeping<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nbookkeeping<\/td>\n<\/tr>\n
shared_ptr<\/tt><\/td>\n\u2192<\/td>\nReturnValue<\/tt><\/td>\n\u00a0<\/td>\nReturnValue<\/tt><\/td>\n<\/tr>\n
Parameter p<\/tt><\/td>\n <\/td>\nother stuff<\/td>\n\u00a0<\/td>\nother stuff<\/td>\n<\/tr>\n
Local l1<\/tt><\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nParameter p<\/tt><\/td>\n<\/tr>\n
temporaries<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nLocal l1<\/tt><\/td>\n<\/tr>\n
 <\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\ntemporaries<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
The code paths diverge significally when the coroutine completes. In our first version, our final_suspend<\/code> allows the coroutine state to be destroyed. Destroying the coroutine state destructs the promise, then the parameters, then the bookkeeping, following the usual C++ rule of destructing in reverse order of construction.<\/p>\n
In our second version, our final_suspend<\/code> keeps the coroutine state alive because the caller has a reference to it via the task. The objects now look like this:<\/p>\n
<\/p>\n\n\n\n\n\n\n\n\n\n
Coroutine state<\/th>\n <\/td>\n <\/td>\n\u00a0<\/td>\nCoroutine state<\/th>\n<\/tr>\n
bookkeeping<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nbookkeeping<\/td>\n<\/tr>\n
shared_ptr<\/tt><\/td>\n\u290d<\/td>\nReturnValue<\/tt><\/td>\n\u00a0<\/td>\nReturnValue<\/tt><\/td>\n<\/tr>\n
Parameter p<\/tt><\/td>\n <\/td>\nother stuff<\/td>\n\u00a0<\/td>\nother stuff<\/td>\n<\/tr>\n
Local l1<\/tt><\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nParameter p<\/tt><\/td>\n<\/tr>\n
temporaries<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nLocal l1<\/tt><\/td>\n<\/tr>\n
 <\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\ntemporaries<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
In the first version, the entire coroutine state has been destroyed and deallocated, leaving only the result_holder<\/code> with the Return\u00adValue<\/code> and a little bit of result holder bookkeeping. But in the second version, the coroutine state is still allocated, with live compiler bookkeeping, a live promise, and the live captured parameter. The live compiler bookeeping says “This coroutine has suspended at its final_suspend<\/code>. If you resume the coroutine, it will self-destruct.”<\/p>\n
In both cases, the result_holder<\/code> (either external or embedded) is kept alive by the task<\/code> that we returned to our caller.<\/p>\n
The caller’s co_await<\/code> copies (in the first case) or moves (in the second case) the result out of the Return\u00adValue<\/code>.<\/p>\n
<\/p>\n\n\n\n\n\n\n\n\n\n
Coroutine state<\/th>\n <\/td>\n <\/td>\n\u00a0<\/td>\nCoroutine state<\/th>\n<\/tr>\n
bookkeeping<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nbookkeeping<\/td>\n<\/tr>\n
shared_ptr<\/tt><\/td>\n\u290d<\/td>\nReturnValue<\/tt><\/td>\n\u00a0<\/td>\n(ReturnValue<\/tt>)<\/td>\n<\/tr>\n
Parameter p<\/tt><\/td>\n <\/td>\nother stuff<\/td>\n\u00a0<\/td>\nother stuff<\/td>\n<\/tr>\n
Local l1<\/tt><\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nParameter p<\/tt><\/td>\n<\/tr>\n
temporaries<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nLocal l1<\/tt><\/td>\n<\/tr>\n
 <\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\ntemporaries<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
There is no change to the first version of our coroutine implementation, because copying the Return\u00adValue<\/code> doesn’t modify the original. But in the second version, moving the Return\u00adValue<\/code> to the caller causes the Return\u00adValue<\/code> inside the embedded result_holder<\/code> to become empty. It is still initialized, but it holds nothing of interest.<\/p>\n
And finally, the caller destructs the task, which causes the last bit to disappear. In the first case, the shared promise state destructs, which means that the result holder bookkeeping and Return\u00adValue<\/code> both destruct, and then the memory for the result_holder<\/code> is freed. In the second case, we go through coroutine state destruction, which as we noted earlier destructs the coroutine state destructs the promise, then the parameters, then the bookkeeping.<\/p>\n
<\/p>\n\n\n\n\n\n\n\n\n\n
Coroutine state<\/th>\n <\/td>\n <\/td>\n\u00a0<\/td>\nCoroutine state<\/th>\n<\/tr>\n
bookkeeping<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nbookkeeping<\/td>\n<\/tr>\n
shared_ptr<\/tt><\/td>\n\u290d<\/td>\nReturnValue<\/tt><\/td>\n\u00a0<\/td>\n(ReturnValue<\/tt>)<\/td>\n<\/tr>\n
Parameter p<\/tt><\/td>\n <\/td>\nother stuff<\/td>\n\u00a0<\/td>\nother stuff<\/td>\n<\/tr>\n
Local l1<\/tt><\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nParameter p<\/tt><\/td>\n<\/tr>\n
temporaries<\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\nLocal l1<\/tt><\/td>\n<\/tr>\n
 <\/td>\n <\/td>\n <\/td>\n\u00a0<\/td>\ntemporaries<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
Next time, we’ll compare the two designs and identify pros and cons.<\/p>\n","protected":false},"excerpt":{"rendered":"
Follow them as they pop into existence and vanish.<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-105078","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"
Follow them as they pop into existence and vanish.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/105078","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=105078"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/105078\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=105078"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=105078"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=105078"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}