{"id":109346,"date":"2024-02-01T07:00:00","date_gmt":"2024-02-01T15:00:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/oldnewthing\/?p=109346"},"modified":"2024-02-24T13:03:33","modified_gmt":"2024-02-24T21:03:33","slug":"20240201-00","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20240201-00\/?p=109346","title":{"rendered":"Using virtual memory placeholders to allocate contiguous address space for multiple purposes"},"content":{"rendered":"

Windows 10 Version 1803 added support for virtual memory placeholders<\/i> which let you reserve address space in a way that can be replaced by another virtual memory allocation.<\/p>\n

Suppose you want to create two adjacent memory mappings of 64KB. Before the advent of placeholders, you would have to do something like this (pseudocode):<\/p>\n

bool retry = true;\r\nwhile (retry) {\r\n    addr = VirtualAlloc(MEM_RESERVE, 128KB);\r\n    if (!addr) fail();\r\n\r\n    VirtualFree(addr, 0, MEM_RELEASE);\r\n\r\n    view1 = MapViewOfFileEx(hMapping1, addr, 64KB);\r\n    if (view1) {\r\n        view2 = MapViewOfFileEx(hMapping2, addr + 64KB, 64KB);\r\n        if (view2) {\r\n            retry = false;\r\n        } else {\r\n            UnmapViewOfFile(view1);\r\n        }\r\n    }\r\n}\r\n<\/pre>\n
(The pseudocode would be a little simpler with RAII types to manage the cleanup, but I did things the manual way.)<\/p>\n
First, we reserve 128KB of contiguous address space, looking for a place we can put our two adjacent 64KB memory blocks. If that fails, then there is no contiguous 128KB memory block, and we give up.<\/p>\n
If it succeeds, then we free that memory and then try to map the two 64KB blocks into the space that we recently freed up. If either one fails (due to a multithreaded race where another thread allocated that address space out from under us), then we unwind all the work we did and start over.<\/p>\n
Virtual memory placeholders let you reserve memory in a way that allows a later memory-mapping operation to take over that address space without you having to free it first. It closes the race window where you have to temporarily free the address space so you can allocate something else.<\/p>\n
The categories of memory functions that support allocating address space into an existing placeholder are currently virtual memory allocation (using new flags added to VirtualAlloc2<\/code>) and file mapping (using new flags added to MapViewOfFile3<\/code>).<\/p>\n
With placeholders, the algorithm for creating adjacent memory mappings is much simpler (still pseudocode):<\/p>\n
addr = VirtualAlloc2(MEM_RESERVE | MEM_RESERVE_PLACEHOLDER, 128KB);\r\nif (!addr) fail();\r\n\r\nVirtualFree(addr, 64KB, MEM_RELEASE | MEM_PRESERVE_PLACEHOLDER);\r\n\r\nview1 = MapViewOfFile3(hMapping1, addr, 64KB, MEM_REPLACE_PLACEHOLDER);\r\nif (!view1) {\r\n    VirtualFree(addr, 0, MEM_RELEASE);\r\n    fail();\r\n}\r\n\r\nview2 = MapViewOfFile3(hMapping2, addr + 64KB, 64KB, MEM_REPLACE_PLACEHOLDER);\r\nif (!view2) {\r\n    VirtualFree(addr, 0, MEM_RELEASE);\r\n    UnmapViewOfFile(view1);\r\n    fail();\r\n}\r\n<\/pre>\n
First, we use the new MEM_RESERVE_PLACEHOLDER<\/code> flag to allocate a 128KB placeholder.<\/p>\n
Next, we use VirtualFree<\/code> with the new MEM_PRESERVE_PLACEHOLDER<\/code> to say that we want to split the original placeholder into two placeholders, splitting at the 64KB mark. This splits the 128KB block into two 64KB blocks.<\/p>\n
Finally, we use Map\u00adView\u00adOf\u00adFile3<\/code> with the new MEM_REPLACE_PLACEHOLDER<\/code> flag to indicate that we want the newly-mapped views to go into a space that currently holds a placeholder. Note that when replacing a placeholder, the new allocation must exactly match the position and size of the existing placeholder. No partial replacements allowed. It’s all or nothing. (If you want to do a partial replacement, then split the placeholder like we did here.)<\/p>\n
There are also flags for merging two adjacent placeholders into one big placeholder, or for freeing virtual memory or file mappings and leaving a placeholder behind. There’s  a full sample in the documentation for Virtual\u00adAlloc2<\/code><\/a>, so I’ll defer to that page.<\/p>\n
Bonus chatter<\/b>: Peter Cooper Jr. pointed out in a comment that I didn’t provide any motivation for placeholders.<\/p>\n
One scenario is the “scatter\/gather” case, where you want to  map multiple files into adjacent blocks so you can treat them as if they were one giant file<\/a>. Another is to simplify implementation of a ring buffer, where you  map the same physical buffer into two adjacent blocks<\/a> so that structures which straddle the boundary do not require special treatment.<\/p>\n","protected":false},"excerpt":{"rendered":"
Can you hold this for a second?<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-109346","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"
Can you hold this for a second?<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/109346","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=109346"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/109346\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=109346"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=109346"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=109346"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}