{"id":10923,"date":"2011-04-15T07:00:00","date_gmt":"2011-04-15T07:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/2011\/04\/15\/lock-free-algorithms-the-trycommithand-off-model\/"},"modified":"2011-04-15T07:00:00","modified_gmt":"2011-04-15T07:00:00","slug":"lock-free-algorithms-the-trycommithand-off-model","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20110415-00\/?p=10923","title":{"rendered":"Lock-free algorithms: The try\/commit\/(hand off) model"},"content":{"rendered":"<p>\nThe last lock-free pattern for this week isn&#8217;t actually lock-free,\nbut it does run without blocking.\n<\/p>\n<p>\nThe pattern for what I&#8217;ll call try\/commit\/(hand off) is more complicated\nthan the other patterns, so I&#8217;ll start off by\ndescribing it in words rather than in code,\nbecause the code tends to make things more complicated.\n<\/p>\n<p>\nFirst, you take the state variable and chop it up into pieces.\nYou need some bits\nto be used as a lock and as a <i>work has been handed off<\/i> flag.\nAnd if the work that has been handed off is complicated,\nyou may need some more bits to remember the details of the handoff.\nA common way of doing this is to use a pointer-sized state variable,\nrequire that the objects being pointed to are suitably aligned,\nand reusing the bottom bits as flags.\nFor example, if you require that the objects be <code>DWORD<\/code>-aligned,\nthen the two bottom bits will always be zero\nand you can reuse them as flags.\n<\/p>\n<p>\nTo perform an operation, you first try to lock the state variable.\nIf you can&#8217;t because the state variable is already locked,\nthen you record the details of the operation in the state variable\nand update it atomically.\n<\/p>\n<p>\nIf you succeed in locking the state variable, then you perform\nthe desired operation, but before you unlock the state variable,\nyou look to see if any work has been handed off.\n(This hand-off work is the result of attempts to perform the operation\nwhile you held the lock.)\nIf there is hand-off work, then you perform that work as well.\nOf course, while you&#8217;re doing that, more\nhand-off work may arrive.\nYou can&#8217;t unlock the state variable until you&#8217;ve\ndrained off all the pent-up hand-off work.\n<\/p>\n<p>\nThe code for this pattern tends to be a tangle of loops since there\nis a lot off backing off and retrying.\nEvery atomic operation is its own loop, draining the hand-off work\nis another loop,\nand\nany time an <code>Interlocked&shy;Compare&shy;Exchange<\/code> fails,\nyou have to undo the work you did and retry&mdash;another loop.\n<\/p>\n<p>\nI trust only about five people in the world to write code\nthat is this advanced, and I&#8217;m not one of them.\nBut just to illustrate the principle (although I will certainly\nget the details wrong), here&#8217;s an implementation of a synchronization-like\nobject which I will call a <code>Group&shy;Wait<\/code> for lack of any other\nname.\nIt has the following operations:\n<\/p>\n<ul>\n<li><code>Add&shy;Wait<\/code>:\n    Register an event handle with the group wait.\n<\/li>\n<li>\n    <code>Signal&shy;All<\/code>:\n    Signals all events that are registered with the group wait.\n    Once an event is signalled, it is automatically unregistered\n    from the group wait.\n    If you want the event to be signalled at the next call to\n    <code>Signal&shy;All<\/code> you have to re-add it.\n<\/li>\n<\/ul>\n<p>\nThe group wait object is just a linked list of\n<code>NODE<\/code>s containing the handles being waited on.\n<\/p>\n<p>\nActually, this type of object doesn&#8217;t need to use the try\/commit\/hand off\nmodel.\nIt can be implemented in a much more straightforward manner by\nhaving <code>Add&shy;Wait<\/code> atomically prepend the node to a list\nand having <code>Signal&shy;All<\/code> atomically steal the list.\nThere are even\n<a HREF=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms684121.aspx\">\nprewritten functions to perform these atomic\nlinked list operations for you<\/a>.\nBut I&#8217;m going to implemented it the complicated way\nfor demonstration purposes.\nIn real life, the code would be much simpler.\n<\/p>\n<p>\nSince the bottom two bits of the pointer must be zero due to alignment,\nwe repurpose them as a lock bit and a signal bit.\nThe lock bit is set when the list is locked,\nand the signal bit is set when a signal was requested but had to be\nhanded off because the list was locked.\n<\/p>\n<pre>\n\/\/ WARNING! IF YOU USE THIS CODE YOU ARE AN IDIOT - READ THE TEXT ABOVE\nstruct NODE;\nNODE *Node(LONG_PTR key) { return reinterpret_cast&lt;NODE*&gt;(key); }\nenum {\n Locked = 1,\n Signalled = 2,\n};\nstruct NODE {\n NODE *pnNext;\n HANDLE hEvent;\n LONG_PTR Key() { return reinterpret_cast&lt;LONG_PTR&gt;(this); }\n NODE *Ptr() { return Node(Key() &amp; ~(Locked | Signalled)); }\n};\n#define NODE_INVALID Node(-1)\nclass GroupWait {\npublic:\n GroupWait() : m_pnRoot(NULL) { }\n ~GroupWait();\n BOOL AddWait(HANDLE hEvent);\n void SignalAll();\nprivate:\n NODE *m_pnRoot;\n};\n<\/pre>\n<p>\nSince I will be viewing the <code>NODE*<\/code> as both a pointer\nand as a bunch of bits (which I call a <i>key<\/i>),\nI created some helper methods to save typing.\n<code>Node<\/code> and <code>Key<\/code> convert back and forth\nbetween node pointers and keys,\nand <code>Ptr<\/code> strips off the tag bits and returns a usable\npointer.\n<\/p>\n<p>\nFor notational purposes, a <code>NODE*<\/code> will be written as\nthe combination <code>p|S|L<\/code> where <code>p<\/code> is a\npointer to the next node, <code>S<\/code> is the signalled bit,\nand <code>L<\/code> is the lock bit.\nThe signalled bit is set to indicate that\nwe need to signal all the nodes in the list\nstarting with the <i>next<\/i> node.\n(Think of the <code>S<\/code> bit\nas being attached to the outgoing arrow.)\nFor example, this linked list:\n<\/p>\n<pre>\n   m_pnRoot\n  +--------+-+-+\n  |   *    |0|1|\n  +---|----+-+-+\n      |\n      v\n  +--------+-+-+---------+\nA |   *    |1|?| hEvent1 |\n  +---|----+-+-+---------+\n      |\n      v\n  +--------+-+-+---------+\nB |   *    |?|?| hEvent2 |\n  +---|----+-+-+---------+\n      |\n      v\n  +--------+-+-+---------+\nC |  NULL  |?|?| hEvent3 |\n  +--------+-+-+---------+\n<\/pre>\n<p>\nrepresents a group wait with three registered event handles.\nThe <code>S<\/code> bit is clear on the root pointer,\nwhich means that\nnobody has yet requested that <code>hEvent1<\/code> be signalled.\nOn the other hand,\nthe <code>S<\/code> bit is set on node&nbsp;A, which means that\nall the events after node&nbsp;A need to be signaled,\nspecifically,\n<code>hEvent2<\/code> and <code>hEvent3<\/code>.\nNote that this means that it doesn&#8217;t matter whether the <code>S<\/code>\nbit is set on nodes B&nbsp;or&nbsp;C; those events are\ngetting set regardless because the <code>S<\/code> bit on node&nbsp;A\nalready requested it.\n(In particular, the <code>S<\/code> bit on the last node is meaningless\nsince there are no nodes which come after it.)\n<\/p>\n<p>\nThe <code>L<\/code> bit is meaningless on all pointers\nother than <code>m_pnRoot<\/code>.\n<\/p>\n<p>\nOkay, let&#8217;s start be adding a handle to the wait list:\n<\/p>\n<pre>\nBOOL GroupWait::AddWait(HANDLE hEvent)\n{\n NODE *pnInsert = new(nothrow) NODE;\n if (pnInsert == NULL) return FALSE;\n pnInsert-&gt;hEvent = hEvent;\n NODE *pn;\n NODE *pnNew;\n do {\n  pn = <a HREF=\"http:\/\/blogs.msdn.com\/b\/oldnewthing\/archive\/2011\/04\/12\/10152296.aspx\">InterlockedReadAcquire<\/a>(&amp;m_pnRoot, NODE_INVALID);\n  pnInsert-&gt;pnNext = pn;\n  pnNew = Node(pnInsert-&gt;Key() | (pn-&gt;Key() &amp; Locked));\n } while (InterlockedCompareExchangeRelease(&amp;m_pnRoot, pnNew, pn) != pn);\n return TRUE;\n}\n<\/pre>\n<p>\nTo add a handle to the wait list, we just prepend it to the linked list,\nbeing careful to propagate the <code>L<\/code> bit into the new pointer\nso we don&#8217;t accidentally release a lock that somebody else took.\nWe add the node with the <code>S<\/code> bit clear on the\ninbound pointer since nobody has\nyet asked for this handle to be signalled.\nAfter setting up the node, we attempt to insert it into the head of the\nlist, and if we can&#8217;t (because somebody else beat us to it),\nthen we restart and try again.\nThis is a standard try\/commit\/try again pattern.\n<\/p>\n<p>\n<b>Exercise<\/b>: Is there an ABA race condition here?\n<\/p>\n<p>\nThe <code>Add&shy;Wait<\/code> method illustrates one extreme case of the\ntry\/commit\/hand off model, where there is really nothing to hand off;\nwe did it all ourselves.\nOf course, this does make other parts of the code trickier since they\nhave to go back and\ndeal with nodes that were added while the list was locked.\n<\/p>\n<p>\nThe nasty part of the code is in <code>Signal&shy;All<\/code>.\nI&#8217;ll present it in pieces.\n<\/p>\n<pre>\nvoid GroupWait::SignalAll()\n{\n NODE *pnCapture;\n NODE *pnNew;\n do {\n  pnCapture = InterlockedReadAcquire(&amp;m_pnRoot, NODE_INVALID);\n  if (pnCapture-&gt;Key() &amp; Locked) {\n   pnNew = Node(pnCapture-&gt;Key() | Signaled);\n  } else {\n   pnNew = Node(Locked);\n  }\n } while (InterlockedCompareExchangeAcquire(&amp;m_pnRoot,\n                              pnNew, pnCapture) != pnCapture);\n if (pnCapture-&gt;Key() &amp; Locked) return;\n ...\n<\/pre>\n<p>\nIf the list is locked, then all we do is try to set the <code>S<\/code> bit\non the root.\nIf the list is not locked, then we try to lock it and simultaneously\ndetach all the nodes by replacing the root pointer with <code>NULL|0|1<\/code>.\nEither way, we perform the operation with the try\/commit\/try again pattern\nuntil we finally get through.\n<\/p>\n<p>\nIf the list was locked,\nthen all we had to do was set the <code>S<\/code> bit on the root pointer.\nSetting the <code>S<\/code> bit on the root pointer\nmeans that all the nodes reachable from this pointer\n(<i>i.e.<\/i>, all nodes after the root, which is all nodes)\nshould be signalled,\nwhich is exactly what we want.\nSince the list is locked, we leave the actual signalling to the code\nthat unlocks the list.\n(This is the <i>hand off<\/i> part of <i>try\/commit\/hand off<\/i>.)\n<\/p>\n<p>\n<b>Exercise<\/b>:\nWhat if the <code>S<\/code> bit is already set?\nDid we lose a signal?\n<\/p>\n<p>\nOtherwise, we are the ones to lock the list.\nWe also detach the node list, for if another thread calls\n<code>Signal&shy;All<\/code>,\nwe don&#8217;t want that signal to affect the nodes that we&#8217;re signalling.\n(Otherwise we might end up double-signalling the event.)\n<\/p>\n<pre>\n ...\n NODE *pnNext;\n NODE *pn;\n for (pn = pnCapture-&gt;Ptr(); pn; pn = pnNext) {\n  SetEvent(pn-&gt;hEvent);\n  pnNext = pn-&gt;pnNext-&gt;Ptr();\n  delete pn;\n }\n ...\n<\/pre>\n<p>\nThat little fragment above is basically what you would do in a\nna&iuml;ve implementation that didn&#8217;t worry about multithreading:\nIt walks the list of nodes, signals each event,\nand then frees the node.\nThe only trick is sending each node pointer through <code>-&gt;Ptr()<\/code>\nto strip off the tag bits.\n<\/p>\n<p>\nNext comes the unlock code.\nFirst, a preparatory step:<\/p>\n<pre>\n ...\n pnCapture = pnNew;\n ...\n<\/pre>\n<p>\nWe exchanged <code>pnNew<\/code> into <code>m_pnRoot<\/code> up above,\nand if that&#8217;s still the value of <code>m_pnRoot<\/code>, then it\nmeans that nobody tried to perform any operations while the list\nwas locked, and we got off easy.\n<\/p>\n<pre>\n ...\n for (;;) {\n  pnNew = Node(pnCapture-&gt;Key() &amp; ~Locked);\n  if (InterlockedCompareExchangeRelease(&amp;m_pnRoot,\n                      pnNew, pnCapture) == pnCapture) {\n   return;\n  }\n ...\n<\/pre>\n<p>\nWe start a new loop whose job is to\ndrain off all the\nhanded-off work items that built up while the list was locked.\nFirst, we see whether anything has changed since the last time\nwe looked; if not, then we unlock and we&#8217;re done.\nOtherwise, we proceed to pick up all the handed-off work:\n<\/p>\n<pre>\n ...\n  pnCapture = InterlockedReadAcquire(&amp;m_pnRoot, NODE_INVALID);\n  NODE *pnNew = Node(pnCapture-&gt;Key() &amp; ~(Locked | Signaled));\n  NODE **ppn = &amp;pnNew;\n  NODE *pn;\n  NODE *pnNext;\n  BOOL fSignalSeen = FALSE;\n  for (pn = pnNew; pn-&gt;Ptr(); pn = pnNext) {\n   pnNext = pn-&gt;Ptr()-&gt;pnNext;\n   if (fSignalSeen) {\n    SetEvent(pn-&gt;Ptr()-&gt;hEvent);\n    delete pn-&gt;Ptr();\n   } else if (pn-&gt;Key() &amp; Signaled) {\n    fSignalSeen = TRUE;\n    (*ppn) = Node(Locked); \/\/ detach but retain lock\n    SetEvent(pn-&gt;Ptr()-&gt;hEvent);\n    delete pn-&gt;Ptr();\n   } else {\n    ppn = &amp;pn-&gt;Ptr()-&gt;pnNext;\n   }\n  }\n } \/\/ retry unlock\n} \/\/ end of function\n<\/pre>\n<p>\nTo drain the handed-off work, we walk the list of nodes,\nkeeping track of whether we&#8217;ve seen an <code>S<\/code> bit.\nIf so, then we signal the event and free the node.\nAnd the first time we see an <code>S<\/code> bit, we null out\nthe inbound pointer to detach the list from the chain so we\ndo not double-signal the event in the future.\n<\/p>\n<p>\nOnce that&#8217;s done, we go back and try to unlock again.\nEventually, there will be no more hand-off work, and we\ncan finally return.\n<\/p>\n<p>\nAnd that&#8217;s it, a demonstration of the try\/commit\/hand off model.\nThe basic idea is simple, but getting all the details right\nis what makes your head hurt.\n<\/p>\n<\/p>\n<p>I leave this sort of thing to the kernel folks, who have the\ntime and patience and brainpower to work it all through.\nAn example of this pattern can be found, for example,\nin this talk that describes the\n<a HREF=\"https:\/\/channel9.msdn.com\/shows\/Going+Deep\/Arun-Kishan-Farewell-to-the-Windows-Kernel-Dispatcher-Lock\/\">\ndismantling of the dispatcher spinlock<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The last lock-free pattern for this week isn&#8217;t actually lock-free, but it does run without blocking. The pattern for what I&#8217;ll call try\/commit\/(hand off) is more complicated than the other patterns, so I&#8217;ll start off by describing it in words rather than in code, because the code tends to make things more complicated. First, you [&hellip;]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-10923","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"<p>The last lock-free pattern for this week isn&#8217;t actually lock-free, but it does run without blocking. The pattern for what I&#8217;ll call try\/commit\/(hand off) is more complicated than the other patterns, so I&#8217;ll start off by describing it in words rather than in code, because the code tends to make things more complicated. First, you [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/10923","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=10923"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/10923\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=10923"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=10923"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=10923"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}