{"id":63,"date":"2014-09-05T07:00:00","date_gmt":"2014-09-05T07:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/2014\/09\/05\/you-can-use-a-file-as-a-synchronization-object-too\/"},"modified":"2014-09-05T07:00:00","modified_gmt":"2014-09-05T07:00:00","slug":"you-can-use-a-file-as-a-synchronization-object-too","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20140905-00\/?p=63","title":{"rendered":"You can use a file as a synchronization object, too"},"content":{"rendered":"<p>\nA customer was looking for a synchronization object that\nhad the following properties:\n<\/p>\n<ul>\n<li>Can be placed in a memory-mapped file.\n<li>Can be used by multiple processes simultaneously.\n    Bonus if it can even be used by different machines simultaneously.<\/p>\n<li>Does not leak resources if the file is deleted.\n<\/ul>\n<p>\nIt turns out there is already a synchronization object for this,\nand you&#8217;ve been staring at it the whole time: The file.\n<\/p>\n<p>\nFile locking is a very old feature that most people consider\nold and busted because it&#8217;s just one of those dorky things\ndesigned for those clunky database systems that use tape drives\nlike they have in the movies.\nWhile that may be true, it&#8217;s still useful.\n<\/p>\n<p>\nThe idea behind file locking is that every byte of a file\ncan be a synchronization object.\nThe intended pattern is that a database program indicates\nits intention to access a section of a file by locking it,\nand this prevents other processes from accessing that same\nsection of the file.\nThis allows the database program to update the file without\nrace conditions.\nWhen the database program is finished with that section of the file,\nit unlocks it.\n<\/p>\n<p>\nOne interesting bit of trivia about file locking is that\nyou can lock bytes that don&#8217;t even exist.\nIt is legal to lock bytes beyond the end of the file.\nThis is handy in the database case if you want to extend the file.\nYou can lock the bytes you intend to add,\nso that nobody else can extend the file at the same time.\n<\/p>\n<p>\nThe usage pattern for byte-granular file locks\nmaps very well to the customer&#8217;s requirements.\nThe synchronization object is&#8230; the file itself.\nAnd you put it in the file by simply choosing a byte to use\nas the lock target.\n(And the byte can even be imaginary.)\nAnd if you delete the file, the lock disappears with it.\n<\/p>\n<p>\nNote that the byte you choose as your lock target need not\nbe dedicated for use as a lock target.\nYou can completely ignore the contents of the file and simply\nagree to use byte zero as the lock target.\nYou just have to understand that when the byte is locked,\nonly the owner of the lock can access it via the\n<code>Read&shy;File<\/code> and <code>Write&shy;File<\/code>\nfamily of functions.\n(Reading or writing a byte that is locked by somebody\nelse will fail with\n<code>ERROR_LOCK_VIOLATION<\/code>.\nNote that access via memory-mapping is not subject to file locking,\nwhich neatly lines up with the customer&#8217;s first requirement.)\n<\/p>\n<p>\nTo avoid the problem with locking an actual byte,\nyou can choose imaginary bytes\nat ridiculously\nhuge offsets purely for locking.\nSince those bytes don&#8217;t exist, you won&#8217;t interfere\nwith other code that tries to read and write them.\nFor example, you might agree to lock byte\n0xFFFFFFFF`FFFFFFFF, on the assumption that the file will never\nbecome four\n<a HREF=\"http:\/\/blogs.msdn.com\/b\/oldnewthing\/archive\/2009\/06\/11\/9725386.aspx\">\nexabytes<\/a> in size.\n<\/p>\n<p>\nFile locking supports the reader\/writer lock model:\nYou can claim a lock for shared access (read)\nor for exclusive access (write).\n<\/p>\n<p>\nThe basic <code>Lock&shy;File<\/code> function is a subset of the\nmore general\n<code>Lock&shy;File&shy;Ex<\/code> function,\nso let&#8217;s look at the general function.\n<\/p>\n<p>\nTo lock a portion of a file, you call\n<code>Lock&shy;File&shy;Ex<\/code> with the range you want to lock,\nthe style of lock (shared or exclusive),\nand how you want failed locks to be handled.\nTo release the lock, you pass the <i>same range<\/i> to\n<code>Unlock&shy;File&shy;Ex<\/code>.\nNote that ranges cannot be chopped up or recombined.\nIf you lock bytes 0&ndash;10 and 11&ndash;19 with separate calls,\nthen you must unlock them with separate matching calls;\nyou can&#8217;t make a single bulk call to unlock bytes\n0&ndash;19, nor can you do a partial unlock of bytes\n0&ndash;5.\n<\/p>\n<p>\nMost of the mechanics of locking are straightforward,\nexcept for the &#8220;how you want failed locks to be handled&#8221; part.\nIf you specify\n<code>LOCKFILE_FAIL_IMMEDIATELY<\/code>\nand the lock attempt fails, then the call simply fails\nwith <code>ERROR_LOCK_VIOLATION<\/code> and that&#8217;s\nthe end of it.\nIt&#8217;s up to you to retry the operation if that&#8217;s what you want.\n<\/p>\n<p>\nOn the other hand, if you do not specify\n<code>LOCKFILE_FAIL_IMMEDIATELY<\/code>,\nand the lock attempt fails,\nthen the behavior depends on whether the handle is\nsynchronous or asynchronous.\nIf synchronous, then the call blocks until the lock is acquired.\nIf asynchronous, then the call returns immediately with\n<code>ERROR_IO_PENDING<\/code>, and the I\/O completes\nwhen the lock is acquired.\n<\/p>\n<p>\nThe documentation in MSDN on how lock failures are handled is\na bit confusing, thanks to tortured sentence structure like\n&#8220;X behaves like Y if Z unless Q.&#8221;\nHere is the behavior of lock failures in table form:\n<\/p>\n<table BORDER=\"1\" STYLE=\"border-collapse: collapse\" CELLSPACING=\"0\" CELLPADDING=\"3\">\n<tr>\n<th ROWSPAN=\"2\">If <code>Lock&shy;File&shy;Ex<\/code> fails<\/th>\n<th COLSPAN=\"2\">Handle type<\/th>\n<\/tr>\n<tr>\n<th>Asynchronous<\/th>\n<th>Synchronous<\/th>\n<\/tr>\n<tr>\n<td><code>LOCKFILE_FAIL_IMMEDIATELY<\/code> specified<\/td>\n<td COLSPAN=\"2\" ALIGN=\"center\">\n        Returns <code>FALSE<\/code> immediately.<br \/>\n        Error code is <code>ERROR_LOCK_VIOLATION<\/code>.\n    <\/td>\n<\/tr>\n<tr>\n<td><code>LOCKFILE_FAIL_IMMEDIATELY<\/code> not specified<\/td>\n<td ALIGN=\"center\">\n        Returns <code>FALSE<\/code> immediately.<br \/>\n        Error code is <code>ERROR_IO_PENDING<\/code>.<br \/>\n        I\/O completes when lock is acquired.<\/td>\n<td ALIGN=\"center\">\n        Blocks until lock is acquired, returns <code>TRUE<\/code>.<\/td>\n<\/table>\n<p>\nHere&#8217;s a little test app that exercises all the options.\nRun the program with two command line options.\nThe first is the name of the file you want to lock,\nand the second is a string describing what kind of lock\nyou want.\nPass zero or more of the following letters:\n<\/p>\n<ul>\n<li>&#8220;o&#8221; to open an overlapped (asynchronous) handle; otherwise,\n    it will be opened non-overlapped (synchronous).<\/p>\n<li>&#8220;e&#8221; to lock exclusively; otherwise, it will be locked shared\n<li>&#8220;f&#8221; to fail immediately; otherwise, it will wait\n<\/ul>\n<p>\nFor example, you would pass &#8220;ef&#8221; to open a synchronous handle\nand request an exclusive lock that fails immediately if it cannot\nbe acquired.\nIf you want all the defaults, then pass &#8220;&#8221; as the options.\n<\/p>\n<pre>\n#include &lt;windows.h&gt;\n#include &lt;stdio.h&gt;\n#include &lt;tchar.h&gt;\nint __cdecl _tmain(int argc, TCHAR **argv)\n{\n \/\/ Ensure correct number of command line arguments\n if (argc &lt; 3) return 0;\n \/\/ Get the options\n DWORD dwFileFlags = 0;\n DWORD dwLockFlags = 0;\n for (PTSTR p = argv[2]; *p; p++) {\n  if (*p == L'o') dwFileFlags |= FILE_FLAG_OVERLAPPED;\n  if (*p == L'e') dwLockFlags |= LOCKFILE_EXCLUSIVE_LOCK;\n  if (*p == L'f') dwLockFlags |= LOCKFILE_FAIL_IMMEDIATELY;\n }\n \/\/ Open the file\n _tprintf(TEXT(\"Opening the file '%s' as %s\\n\"), argv[1],\n          (dwFileFlags &amp; FILE_FLAG_OVERLAPPED) ?\n          TEXT(\"asynchronous\") : TEXT(\"synchronous\"));\n HANDLE h = CreateFile(argv[1], GENERIC_READ,\n                FILE_SHARE_READ | FILE_SHARE_WRITE,\n                NULL, OPEN_EXISTING,\n                FILE_ATTRIBUTE_NORMAL | dwFileFlags, NULL);\n if (h == INVALID_HANDLE_VALUE) {\n  _tprintf(TEXT(\"Open failed, error = %d\\n\"), GetLastError());\n  return 0;\n }\n \/\/ Set the starting position in the OVERLAPPED structure\n OVERLAPPED o = { 0 };\n o.Offset = 0; \/\/ we lock on byte zero\n \/\/ Say what kind of lock we want\n if (dwLockFlags &amp; LOCKFILE_EXCLUSIVE_LOCK) {\n  _tprintf(TEXT(\"Requesting exclusive lock\\n\"));\n } else {\n  _tprintf(TEXT(\"Requesting shared lock\\n\"));\n }\n \/\/ Say whether we're going to wait to acquire\n if (dwLockFlags &amp; LOCKFILE_FAIL_IMMEDIATELY) {\n  _tprintf(TEXT(\"Requesting immediate failure\\n\"));\n } else if (dwFileFlags &amp; FILE_FLAG_OVERLAPPED) {\n  _tprintf(TEXT(\"Requesting notification on lock acquisition\\n\"));\n  \/\/ The event that will be signaled when the lock is acquired\n  \/\/ error checking deleted for expository purposes\n  o.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);\n } else {\n  _tprintf(TEXT(\"Call will block until lock is acquired\\n\"));\n }\n \/\/ Okay, here we go.\n _tprintf(TEXT(\"Attempting lock\\n\"));\n BOOL fRc = LockFileEx(h, dwLockFlags, 0, 1, 0, &amp;o);\n \/\/ If the lock failed, remember why.\n DWORD dwError = fRc ? ERROR_SUCCESS : GetLastError();\n _tprintf(TEXT(\"Wait %s, error code %d\\n\"),\n          fRc ? TEXT(\"succeeded\") : TEXT(\"failed\"), dwError);\n if (fRc) {\n  _tprintf(TEXT(\"Lock acquired immediately\\n\"));\n } else if (dwError == ERROR_IO_PENDING) {\n  _tprintf(TEXT(\"Waiting for lock\\n\"));\n  WaitForSingleObject(o.hEvent, INFINITE);\n  fRc = TRUE; \/\/ lock has been acquired\n }\n \/\/ If we got the lock, then hold the lock until the\n \/\/ user releases it.\n if (fRc) {\n  _tprintf(TEXT(\"Hit Enter to unlock\\n\"));\n  getchar();\n  UnlockFileEx(h, 0, 1, 0, &amp;o);\n }\n \/\/ Clean up\n if (o.hEvent) CloseHandle(o.hEvent);\n CloseHandle(h);\n return 0;\n}\n<\/pre>\n<p>\nWhen you run this program, it will try to acquire\nthe lock in the manner requested,\nand if the lock is successfully acquired,\nit will wait for you press Enter,\nthen it will release the lock.\n<\/p>\n<p>\nYou naturally need to run multiple copies of this program\nto see how the flags interact.\n(If you run only one copy, then it will always succeed.)\n<\/p>\n<p>\n<b>Exercise<\/b>:\nWhat changes would you make if you wanted\nto wait at most 5 seconds to acquire the lock?\n(<a HREF=\"http:\/\/blogs.msdn.com\/b\/oldnewthing\/archive\/2011\/02\/02\/10123392.aspx\">Hint<\/a>.)<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A customer was looking for a synchronization object that had the following properties: Can be placed in a memory-mapped file. Can be used by multiple processes simultaneously. Bonus if it can even be used by different machines simultaneously. Does not leak resources if the file is deleted. It turns out there is already a synchronization [&hellip;]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-63","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"<p>A customer was looking for a synchronization object that had the following properties: Can be placed in a memory-mapped file. Can be used by multiple processes simultaneously. Bonus if it can even be used by different machines simultaneously. Does not leak resources if the file is deleted. It turns out there is already a synchronization [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/63","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=63"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/63\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=63"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=63"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=63"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}