{"id":104093,"date":"2020-08-19T07:00:00","date_gmt":"2020-08-19T14:00:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/oldnewthing\/?p=104093"},"modified":"2020-09-09T06:12:37","modified_gmt":"2020-09-09T13:12:37","slug":"20200819-00","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20200819-00\/?p=104093","title":{"rendered":"On using FILE_FLAG_WRITE_THROUGH and FILE_FLAG_NO_BUFFERING for memory-mapped files"},"content":{"rendered":"

A customer wanted to use the FILE_<\/code>FLAG_<\/code>WRITE_<\/code>THROUGH<\/code> and FILE_<\/code>FLAG_<\/code>NO_<\/code>BUFFERING<\/code> flags for a memory-mapped file, based on this guidance in the documentation for Create\u00adFile<\/code>:<\/p>\n

\n

For this reason, the FILE_FLAG_WRITE_THROUGH<\/b> flag is often used with the FILE_FLAG_NO_BUFFERING<\/b> flag as a replacement for calling the FlushFileBuffers function after each write, which can cause unnecessary performance penalties. Using these flags together avoids those penalties.<\/p>\n<\/blockquote>\n

The customer was concerned whether this combination of flags will affect data consistency.<\/p>\n

Actually, the customer’s problems with data consistency started even before they got around to worrying about these flags.<\/p>\n

Since they are using a memory-mapped file, they don’t have any direct control over when the memory gets written to disk. Page from memory-mapped files are written to disk at the operating system’s discretion. Therefore, if they write information into two pages of a memory-mapped file, the pages can be written to disk in any order.<\/p>\n

Since they’re asking about data consistency, they must be worried about power loss or system crashes before the data can be written to disk. And since the pages can be written in either order, all four outcomes of two dirty pages are possible.<\/p>\n\n\n\n\n\n\n\n
Page 1 written to disk<\/th>\nPage 2 written to disk<\/th>\n<\/tr>\n
No<\/td>\nNo<\/td>\n<\/tr>\n
Yes<\/td>\nNo<\/td>\n<\/tr>\n
No<\/td>\nYes<\/td>\n<\/tr>\n
Yes<\/td>\nYes<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

So much for data consistency.<\/p>\n

Setting those flags on a memory-mapped file controls how<\/i> the operating system writes the memory to disk, but it doesn’t provide any control over when<\/i> the memory is written to disk. And without that control, you don’t really have data consistency.<\/p>\n

Usually, when designing a system for consistency, you have a specific order in which data needs to be written to the disk. For example, you might decide to write the new data to the disk, and then once that’s safe, you write new metadata (say, by updating an index) that causes the new data to become the active values, and the old data to be ignored. Those are the writes that would be able to take advantage of the write-through and buffering flags.<\/p>\n

Bonus chatter<\/b>: Using FILE_FLAG_NO_BUFFERING<\/code> with a memory-mapped file doesn’t really serve any purpose. The “no buffering” flag means that the writes go straight to the disk without being cached in memory. But the whole point of a memory-mapped file is to be cached in memory!<\/p>\n","protected":false},"excerpt":{"rendered":"

You’re trying to control I\/O you didn’t issue.<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-104093","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"

You’re trying to control I\/O you didn’t issue.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/104093","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=104093"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/104093\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=104093"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=104093"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=104093"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}