{"id":795,"date":"2017-09-06T12:31:47","date_gmt":"2017-09-06T04:31:47","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/seteplia\/?p=795"},"modified":"2019-06-11T21:06:16","modified_gmt":"2019-06-12T04:06:16","slug":"managed-object-internals-part-2-object-header-layout-and-the-cost-of-locking","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/premier-developer\/managed-object-internals-part-2-object-header-layout-and-the-cost-of-locking\/","title":{"rendered":"Managed object internals, Part 2. Object header layout and the cost of locking"},"content":{"rendered":"<p>Working on my current project I\u2019ve faced a very interesting situation. For each object of a given type, I had to create a monotonically growing identifier with few caveats: 1) the solution should work in multithreaded environment 2) the number of objects is fairly large, up to 10 million instances and 3) identity should be created lazily because not every object needs it.<\/p>\n<p>During my original implementation, I haven\u2019t realized the amount of instance the application will deal with, so I came up with a very simple solution:<\/p>\n<pre class=\"lang:default decode:true \">public class Node\r\n{\r\n    public const int InvalidId = -1;\r\n    private static int s_idCounter;\r\n\r\n    private int m_id;\r\n\r\n    public int Id\r\n    {\r\n        get\r\n        {\r\n            if (m_id == InvalidId)\r\n            {\r\n                lock (this)\r\n                {\r\n                    if (m_id == InvalidId)\r\n                    {\r\n                        m_id = Interlocked.Increment(ref s_idCounter);\r\n                    }\r\n                }\r\n            }\r\n\r\n            return m_id;\r\n        }\r\n    }\r\n}<\/pre>\n<p><a name=\"OLE_LINK2\"><\/a>The code is using <a href=\"https:\/\/en.wikipedia.org\/wiki\/Double-checked_locking\">double-checked locking pattern<\/a> that allows initializing an identity field lazily in multithreaded environment. During one of the profiling sessions, I\u2019ve noticed that the number of objects with a valid id reached few million instances and the main surprise was that it didn\u2019t cause any issues in terms of performance.<\/p>\n<p>After that, I\u2019ve created a benchmark to see the impact of the <b>lock<\/b> statement in terms of performance compared to a no-lock approach.<\/p>\n<pre class=\"lang:default decode:true\">public class NoLockNode\r\n{\r\n    public const int InvalidId = -1;\r\n    private static int s_idCounter;\r\n\r\n    private int m_id = InvalidId;\r\n\r\n    public int Id\r\n    {\r\n        get\r\n        {\r\n            if (m_id == InvalidId)\r\n            {\r\n                \/\/ Leaving double check to have the same amount of computation here\r\n                if (m_id == InvalidId)\r\n                {\r\n                    m_id = Interlocked.Increment(ref s_idCounter);\r\n                }\r\n            }\r\n\r\n            return m_id;\r\n        }\r\n    }\r\n}<\/pre>\n<p>To analyze performance differences, I\u2019ll be using <a href=\"http:\/\/benchmarkdotnet.org\/Overview.htm\">BenchmarkDotNet<\/a>:<\/p>\n<pre class=\"lang:default decode:true \">List&lt;NodeWithLock.Node&gt; m_nodeWithLocks =&gt; \r\n    Enumerable.Range(1, Count).Select(n =&gt; new NodeWithLock.Node()).ToList();\r\nList&lt;NodeNoLock.NoLockNode&gt; m_nodeWithNoLocks =&gt; \r\n    Enumerable.Range(1, Count).Select(n =&gt; new NodeNoLock.NoLockNode()).ToList();\r\n\r\n[Benchmark]\r\npublic long NodeWithLock()\r\n{\r\n    \/\/ m_nodeWithLocks has 5 million instances\r\n    return m_nodeWithLocks\r\n        .AsParallel()\r\n        .WithDegreeOfParallelism(16)\r\n        .Select(n =&gt; (long)n.Id).Sum();\r\n}\r\n\r\n[Benchmark]\r\npublic long NodeWithNoLock()\r\n{\r\n    \/\/ m_nodeWithNoLocks has 5 million instances\r\n    return m_nodeWithNoLocks\r\n        .AsParallel()\r\n        .WithDegreeOfParallelism(16)\r\n        .Select(n =&gt; (long)n.Id).Sum();\r\n}<\/pre>\n<p>In this case, <b>NoLockNode<\/b> is not suitable for multithreaded scenarios, but our benchmark doesn\u2019t try to get <b>Id<\/b>\u2019s for two instances from different threads at the same time either. The benchmark mimics the real-world scenario when contention is happening rarely and in most cases the application just consumes already created identifier.<\/p>\n<pre class=\"lang:default decode:true \">                      Method |        Mean |    StdDev |\r\n---------------------------- |------------ |---------- |\r\n                NodeWithLock | 152.2947 ms | 1.4895 ms |\r\n              NodeWithNoLock | 149.5015 ms | 2.7289 ms |<\/pre>\n<p>As we can see the difference is extremely small. How can the CLR get 1 million locks with almost 0 overhead?<\/p>\n<p>To shed some light on the CLR behavior, let\u2019s extend our benchmarking test suite by another case. Let\u2019s add another <b>Node<\/b> class that calls <b>GetHashCode<\/b> method in the constructor (non overridden version of it) and then discards the result:<\/p>\n<pre class=\"lang:default decode:true \">public class Node\r\n{\r\n    public const int InvalidId = -1;\r\n    private static int s_idCounter;\r\n    private object syncRoot = new object();\r\n    private int m_id = InvalidId;\r\n    public Node()\r\n    {\r\n        GetHashCode();\r\n    }\r\n\r\n    public int Id\r\n    {\r\n        get\r\n        {\r\n            if (m_id == InvalidId)\r\n            {\r\n                lock(this)\r\n                {\r\n                    if (m_id == InvalidId)\r\n                    {\r\n                        m_id = Interlocked.Increment(ref s_idCounter);\r\n                    }\r\n                }\r\n            }\r\n\r\n            return m_id;\r\n        }\r\n    }\r\n}<\/pre>\n<pre class=\"lang:default decode:true  \">                      Method |        Mean |    StdDev |\r\n---------------------------- |------------ |---------- |\r\n                NodeWithLock | 152.2947 ms | 1.4895 ms |\r\n              NodeWithNoLock | 149.5015 ms | 2.7289 ms |\r\n  NodeWithLockAndGetHashCode | 541.6314 ms | 4.0445 ms |<\/pre>\n<p>The result of the <b>GetHashCode<\/b> invocation is discarded and the call itself does not affect the end-2-end time because the benchmark excludes construction time from the measurements. But the questions are: why there is almost 0 overhead for a <b>lock<\/b> statement in case of <b>NodeWithLock<\/b> and significant difference when the <b>GetHashCode<\/b> method was called on the object instance in <b>NodeWithLockAndGetHashCode<\/b>?<\/p>\n<h4>Thin locks, inflation and object header layout<\/h4>\n<p>Each object in the CLR can be used to create a critical region to achieve mutually exclusive execution. And you may think that to do so, the CLR creates a kernel object for each CLR object. But this approach would make no sense just because only a very small fraction of all the objects are being used as handles for synchronization purposes. So, it makes a perfect sense that the CLR creates expensive data structures required for synchronization lazily. Moreover, if the CLR can avoid redundant data structures, it won\u2019t create them at all.<\/p>\n<p>As you may know, every managed object has an auxiliary field for every object called the object header. The header itself can be used for different purposes and can keep different information based on the current object\u2019s state.<\/p>\n<p>The CLR can store object\u2019s hash code, domain specific information, lock-related data and some other stuff at the same time. Apparently, 4 bytes of the object header is simply not enough for all of that. So, the CLR will create an auxiliary data structure called sync block table and will keep just an index in the header itself. But the CLR will try to avoid that and will try to put as much data in the header itself as possible.<\/p>\n<p>Here is a layout for a most significant byte of the object header:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/06\/clip_image0022.png\"><img decoding=\"async\" style=\"padding-top: 0px; padding-left: 0px; padding-right: 0px; border-width: 0px;\" title=\"clip_image002\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/06\/clip_image002_thumb2.png\" alt=\"clip_image002\" width=\"1231\" height=\"386\" border=\"0\" \/><\/a><\/p>\n<p>If the BIT_SBLK_IS_HASH_OR_SYNCBLKINDEX bit is 0, then the header itself keeps all lock-related information and the lock is called a \u201cthin lock\u201d. In this case, the overall layout of the object header is the following:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/06\/image35.png\"><img decoding=\"async\" style=\"padding-top: 0px; padding-left: 0px; padding-right: 0px; border: 0px;\" title=\"image\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/06\/image_thumb24.png\" alt=\"image\" width=\"2180\" height=\"445\" border=\"0\" \/><\/a><\/p>\n<p>If the BIT_SBLK_IS_HASH_OR_SYNCBLKINDEX bit is 1 then the sync block created for an object or the hash code was computed. If the BIT_SBLK_IS_HASHCODE is 1 (bit #26) then the rest of the dword (lower 26 bits) is the object\u2019s hash code, otherwise, the lower 26 bits represents a sync block index:<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/06\/image36.png\"><img decoding=\"async\" style=\"padding-top: 0px; padding-left: 0px; padding-right: 0px; border: 0px;\" title=\"image\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/31\/2019\/06\/image_thumb25.png\" alt=\"image\" width=\"2177\" height=\"793\" border=\"0\" \/><\/a><\/p>\n<p>We can investigate thin locks using WinDbg with SoS extension. First, we\u2019ll stop the execution inside a lock statement for a simple object instance that doesn\u2019t call a <b>GetHashCode<\/b> method:<\/p>\n<pre class=\"lang:default decode:true \">object o = new object();\r\nlock (o)\r\n{\r\n    Debugger.Break();\r\n}<\/pre>\n<p>Inside WinDbg we\u2019ll run <b>.loadby sos clr<\/b> to get <a href=\"https:\/\/docs.microsoft.com\/en-us\/dotnet\/framework\/tools\/sos-dll-sos-debugging-extension\">SOS Debugging extension<\/a> and then two commands: <b>!DumpHeap -thinlock<\/b> to see all thin locks and then <b>!DumpObj<\/b> <b>obj<\/b> to see the state of the instance we\u2019re using in the lock statement:<\/p>\n<p><span style=\"font-family: Consolas;\">0:000&gt; !DumpHeap -thinlock <\/span><\/p>\n<p><span style=\"font-family: Consolas;\">Address MT Size <\/span><\/p>\n<p><span style=\"font-family: Consolas;\"><b>02d223e0 725c2104 12 ThinLock owner 1 (00ea5498) Recursive 0<\/b> <\/span><\/p>\n<p><span style=\"font-family: Consolas;\"><b>Found 1 objects.<\/b> <\/span><\/p>\n<p><span style=\"font-family: Consolas;\">0:000&gt; !DumpObj \/d 02d223e0 <\/span><\/p>\n<p><span style=\"font-family: Consolas;\">Name: System.Object <\/span><\/p>\n<p><span style=\"font-family: Consolas;\">MethodTable: 725c2104 <\/span><\/p>\n<p><span style=\"font-family: Consolas;\"><b>ThinLock owner 1 (00ea5498), Recursive 0<\/b> <\/span><\/p>\n<p>There are at least two cases when the thin lock can be promoted to a \u201cfat lock\u201d: (1) contention on the sync root from another thread that will require a kernel object to be created or (2) inability for the CLR to keep all the information in the object header, for instance, because of a call to a <b>GetHashCode<\/b> method.<\/p>\n<p>The CLR monitor is implemented as a \u201chybrid lock\u201d that tries to spin first before creating a real Win32 kernel object. Here is a short description of a monitor from the \u201cConcurrent Programming on Windows\u201d by Joe Duffy: \u201cOn a single-CPU machine, the monitor implementation will do a scaled back spin-wait: the current thread&#8217;s timeslice is yielded to the scheduler several times by calling SwitchToThread before waiting. On a multi-CPU machine, the monitor yields the thread every so often, but also busy-spins for a period of time before falling back to a yield, using an exponential back-off scheme to control the frequency at which it rereads the lock state. All of this is done to work well on Intel HyperThreaded machines. If the lock still is not available after the fixed spin wait period has been exhausted, the acquisition attempt falls back to a true wait using an underlying Win32 event. We discuss how this works in a bit.\u201d<\/p>\n<p>We can check that in both cases the inflation is indeed happening and a thin lock is promoted to a fat lock:<\/p>\n<pre class=\"lang:default decode:true \">object o = new object();\r\n\/\/ Just need to call GetHashCode and discard the result\r\no.GetHashCode();\r\nlock (o)\r\n{\r\n    Debugger.Break();\r\n}<\/pre>\n<p><span style=\"font-family: Consolas;\">0:000&gt; !dumpheap -thinlock <\/span><\/p>\n<p><span style=\"font-family: Consolas;\">Address MT Size <\/span><\/p>\n<p><span style=\"font-family: Consolas;\"><b>Found 0 objects.<\/b> <\/span><\/p>\n<p><span style=\"font-family: Consolas;\">0:000&gt; <b>!syncblk<\/b> <\/span><\/p>\n<p><span style=\"font-family: Consolas;\">Index SyncBlock MonitorHeld Recursion Owning Thread Info SyncBlock Owner <\/span><\/p>\n<p><span style=\"font-family: Consolas;\">1 011790a4 1 1 01155498 4ea8 0 02db23e0 System.Object <\/span><\/p>\n<p>As you can see, just by calling <b>GetHashCode<\/b> on a synchronization object we\u2019ll get a different result. Now there is no thin locks and the sync root has a sync block associated with it.<\/p>\n<p>We can get the same result if the other thread will content for a long period of time:<\/p>\n<pre class=\"lang:default decode:true \">object o = new object();\r\nlock (o)\r\n{\r\n    Task.Run(() =&gt;\r\n    {\r\n        \/\/ This will promote a thin lock as well\r\n        lock (o) { }\r\n    });\r\n\r\n    \/\/ 10 ms is not enough, the CLR spins longer than 10 ms.\r\n    Thread.Sleep(100);\r\n    Debugger.Break();\r\n}<\/pre>\n<p>In this case, the result would be the same: a thin lock would be promoted and a sync block entry would be created.<\/p>\n<p><span style=\"font-family: Consolas;\">0:000&gt; !dumpheap -thinlock <\/span><\/p>\n<p><span style=\"font-family: Consolas;\">Address MT Size <\/span><\/p>\n<p><span style=\"font-family: Consolas;\"><b>Found 0 objects.<\/b> <\/span><\/p>\n<p><span style=\"font-family: Consolas;\">0:000&gt; !syncblk <\/span><\/p>\n<p><span style=\"font-family: Consolas;\">Index SyncBlock MonitorHeld Recursion Owning Thread Info SyncBlock Owner <\/span><\/p>\n<p><span style=\"font-family: Consolas;\">6 00d9b378 3 1 00d75498 1884 0 02b323ec System.Object <\/span><\/p>\n<h4>Conclusion<\/h4>\n<p>Now the benchmark output should be easier to understand. The CLR can acquire millions of locks with close to 0 overhead if it can use thin locks. Thin lock is extremely efficient. To acquire the lock the CLR will change a few bits in the object header to store a thread id there, and the waiting thread will spin until those bits are non-zero. On the other hand, if the thin lock is promoted to a \u201cfat lock\u201d than the overhead becomes more noticeable. Especially if the number of objects acquired a fat lock is fairly large.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Working on my current project I\u2019ve faced a very interesting situation. For each object of a given type, I had to create a monotonically growing identifier with few caveats: 1) the solution should work in multithreaded environment 2) the number of objects is fairly large, up to 10 million instances and 3) identity should be [&hellip;]<\/p>\n","protected":false},"author":4004,"featured_media":37840,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[6700],"tags":[6695],"class_list":["post-795","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-net-internals","tag-seteplia"],"acf":[],"blog_post_summary":"<p>Working on my current project I\u2019ve faced a very interesting situation. For each object of a given type, I had to create a monotonically growing identifier with few caveats: 1) the solution should work in multithreaded environment 2) the number of objects is fairly large, up to 10 million instances and 3) identity should be [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts\/795","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/users\/4004"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/comments?post=795"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/posts\/795\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/media\/37840"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/media?parent=795"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/categories?post=795"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/premier-developer\/wp-json\/wp\/v2\/tags?post=795"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}