{"id":105999,"date":"2021-12-03T07:00:00","date_gmt":"2021-12-03T15:00:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/oldnewthing\/?p=105999"},"modified":"2021-12-03T07:18:17","modified_gmt":"2021-12-03T15:18:17","slug":"20211203-00","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20211203-00\/?p=105999","title":{"rendered":"How can I produce a C-style array from a Windows Runtime asynchronous operation?"},"content":{"rendered":"<p>C-style arrays fall through the cracks of the Windows Runtime. We spent the past few days working around the inability to pass them with transfer semantics. Today we look at another hole, namely the inability for them to be the product of an asynchronous operation.<\/p>\n<p>There is no facility in the Windows Runtime for an <code>IAsync\u00adOperation&lt;T[]&gt;<\/code>. An asynchronous operation can produce a primitive type, a structure type, or a reference type, but not an array.<\/p>\n<p>As with the inability to transfer ownership of a C-style array, we can work around this by producing an <code>IBuffer<\/code>, assuming that the underlying type of the array has no destructor. It suffers from the same awkwardness of getting the data into and out of the buffer, as well as limiting yourself to languages that support raw pointers.<\/p>\n<p>You might try wrapping the array inside a <code>Property\u00adValue<\/code> and using <code>Property\u00adValue.<wbr \/>Get\u00adInt32\u00adArray<\/code> to retrieve it. However, this returns a copy of the underlying array, which can be a problem if the array is large and you&#8217;re trying to avoid copies.<\/p>\n<p>You could create your own wrapper type whose method for producing the C-style array is destructive:<\/p>\n<pre>namespace Sample\r\n{\r\n    runtimeclass WidgetIndicesHolder\r\n    {\r\n        Int32[] DetachIndexArray();\r\n    }\r\n\r\n    runtimeclass Widget\r\n    {\r\n        \/\/ The \"indices\" array will be very large.\r\n        Windows.Foundation.IAsyncOperation&lt;WidgetIndicesHolder&gt;\r\n            GetIndicesAsync();\r\n    }\r\n}\r\n<\/pre>\n<p>The consuming code would do this:<\/p>\n<pre>auto holder = co_await widget.GetIndicesAsync();\r\nauto indices = holder.DetachIndexArray();\r\n<\/pre>\n<p>Another idea is to use the role reversal technique we came up with when dealing with ownership transfer and have the operation complete with a delegate, and then you call the delegate to get the C-style array.<\/p>\n<pre>namespace Sample\r\n{\r\n    <span style=\"color: blue;\">delegate Int32[] WidgetIndicesProducer();<\/span>\r\n\r\n    runtimeclass Widget\r\n    {\r\n        \/\/ The \"indices\" array will be very large.\r\n        Windows.Foundation.IAsyncOperation&lt;<span style=\"color: blue;\">WidgetIndicesProducer<\/span>&gt;\r\n            GetIndicesAsync();\r\n    }\r\n}\r\n<\/pre>\n<p>In this case, the consuming code would look like this:<\/p>\n<pre>auto producer = co_await widget.GetIndicesAsync();\r\nauto indices = producer();\r\n<\/pre>\n<p>Before digging into the implementation, let&#8217;s compare these two strategies.<\/p>\n<p>That&#8217;s a trick question. The two strategies are effectively the same!<\/p>\n<p>The similarly is clearer if I rewrite the delegate in terms of what it looks like at the ABI layer:<\/p>\n<pre>namespace Sample\r\n{\r\n    <span style=\"color: blue;\">runtimeclass WidgetIndicesProducer\r\n    {\r\n        Int32[] Invoke();\r\n    }<\/span>\r\n\r\n    runtimeclass Widget\r\n    {\r\n        \/\/ The \"indices\" array will be very large.\r\n        Windows.Foundation.IAsyncOperation&lt;WidgetIndicesProducer&gt;\r\n            GetIndicesAsync();\r\n    }\r\n}\r\n<\/pre>\n<p>A delegate is just a class with a single <code>Invoke<\/code> method whose parameters are the delegate parameters and whose return type is the delegate return type.\u00b9 The language projection exposes the <code>Invoke<\/code> method as if it were a function call.<\/p>\n<p>As a result, all that we did was rename <code>Widget\u00adIndices\u00adHolder<\/code> to <code>Widget\u00adIndices\u00adProducer<\/code> and <code>Detach\u00adIndex\u00adArray<\/code> to <code>Invoke<\/code>.<\/p>\n<p>So let&#8217;s use the delegate version, since it involves less typing, and it also allows us to reuse the <code>Widget\u00adIndices\u00adProducer<\/code> delegate that we used to transfer a C-style array into a Windows Runtime class.<\/p>\n<p>On the consuming side, we can avoid the <code>producer<\/code> temporary by invoking the returned delegate immediately.<\/p>\n<pre>auto indices = (co_await widget.GetIndicesAsync())();\r\n<\/pre>\n<p>On the producing side, we wrap our return value inside the same sort of delegate we used when we looked at transferring ownership into a class: We move the C-style array into the delegate, and then move it out on request.<\/p>\n<pre>namespace winrt::Sample::implementation\r\n{\r\n    struct Widget : WidgetT&lt;Widget&gt;\r\n    {\r\n        IAsyncOperation&lt;WidgetIndicesProducer&gt;\r\n        GetIndicesAsync()\r\n        {\r\n            auto result = co_await CalculateIndicesAsync();\r\n            co_return WidgetIndicesProducer(\r\n                [result = std::move(result)]() mutable\r\n                { return std::move(result); });\r\n        }\r\n    };\r\n}\r\n<\/pre>\n<p>We can take advantage of the delegate constructor that takes a lambda and avoid having to repeat the name of the delegate:<\/p>\n<pre>namespace winrt::Sample::implementation\r\n{\r\n    struct Widget : WidgetT&lt;Widget&gt;\r\n    {\r\n        IAsyncOperation&lt;WidgetIndicesProducer&gt;\r\n        GetIndicesAsync()\r\n        {\r\n            auto result = ... calculate the indices ...\r\n            co_return\r\n                [result = std::move(result)]() mutable\r\n                { return std::move(result); };\r\n        }\r\n    };\r\n}\r\n<\/pre>\n<p>Note that it is essential that the C-style array be captured by value into the delegate. The <code>[&amp;]<\/code> capture is definitely wrong, because the delegate is going to outlive the call to <code>Get\u00adIndices\u00adAsync<\/code>.<\/p>\n<p><b>Bonus chatter<\/b>: One customer tried this:<\/p>\n<pre>namespace Sample\r\n{\r\n    runtimeclass Widget\r\n    {\r\n        \/\/ The \"indices\" array will be very large.\r\n        Windows.Foundation.IAsyncAction GetIndicesAsync(out Int32[] indices);\r\n    }\r\n}\r\n<\/pre>\n<p>The idea here is that the caller passes in a variable to receive the indices, but the indices don&#8217;t actually show up until the <code>IAsyncAction<\/code> completes. The intended calling usage would be something like<\/p>\n<pre>winrt::com_array&lt;int32_t&gt; indices;\r\nco_await widget.GetIndicesAsync(indices);\r\n\/\/ use the indices\r\n<\/pre>\n<p>Breaking it down a bit more:<\/p>\n<pre>winrt::com_array&lt;int32_t&gt; indices;\r\nauto action = widget.GetIndicesAsync(indices);\r\n\/\/ indices not yet ready\r\nco_await action;\r\n\/\/ okay, now we have indices\r\n<\/pre>\n<p>This doesn&#8217;t work because the <code>[out]<\/code> parameters are valid only for the lifetime of the call. And the call is done when it returns an <code>IAsyncAction<\/code>.<\/p>\n<p>The <code>indices<\/code> may no longer exist by the time the operation completes. For example, during the &#8220;indices not yet ready&#8221; comment, the caller might decide to go off and do something else, and that other thing might throw an exception, causing everything to unwind and the <code>indices<\/code> to disappear. Or maybe the caller did a <code>wait_for()<\/code> to wait for the indices with a timeout, and if the operation times out, it just gives up. Or maybe the <code>co_await<\/code> threw an exception when trying to register the continuation.<\/p>\n<p>The implementation of <code>GetIndicesAsync<\/code> doesn&#8217;t know that any of these things have happened, and it will happily write to an already-destroyed object, which is a great source of memory corruption.<\/p>\n<p>For garbage-collected languages, it&#8217;s even worse, because even in the absence of errors, garbage collection may run while the <code>IAsyncAction<\/code> is pending. The garbage collector might move the <code>out<\/code> parameter or even destroy it completely if the variable is not used after the <code>await<\/code>.<\/p>\n<p>And certainly it&#8217;s not going to work for marshalled calls, because the server-side implementation receives a server-side <code>indices<\/code> variable which is marshalled back to the client side when the <code>GetIndicesAsync<\/code> function returns an <code>IAsyncAction<\/code>. COM doesn&#8217;t know that &#8220;Oh, wait, don&#8217;t marshal it back yet. I&#8217;m going to update it some more later.&#8221;<\/p>\n<p>So don&#8217;t do that. It doesn&#8217;t work and corrupts memory.<\/p>\n<p>\u00b9 This isn&#8217;t strictly true, but it&#8217;s true enough for the purpose of this discussion. Another difference is that delegates derive directly from <code>IUnknown<\/code> rather than from <code>IInspectable<\/code>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>No good solutions, but some workarounds.<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-105999","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"<p>No good solutions, but some workarounds.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/105999","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=105999"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/105999\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=105999"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=105999"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=105999"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}