{"id":21033,"date":"2008-08-29T10:00:00","date_gmt":"2008-08-29T10:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/2008\/08\/29\/what-possible-use-are-those-extra-bits-in-kernel-handles-part-3-new-object-types\/"},"modified":"2008-08-29T10:00:00","modified_gmt":"2008-08-29T10:00:00","slug":"what-possible-use-are-those-extra-bits-in-kernel-handles-part-3-new-object-types","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20080829-00\/?p=21033","title":{"rendered":"What possible use are those extra bits in kernel handles? Part 3: New object types"},"content":{"rendered":"<p><P>\nLast time, we saw how\n<A HREF=\"http:\/\/blogs.msdn.com\/oldnewthing\/archive\/2008\/08\/28\/8902173.aspx\">\nthose extra bits can be used to multiplex <CODE>HANDLE<\/CODE>\nwith other values<\/A>.\nThat was a specific case of a more general scenario:\nExpanding the handle namespace to include things that aren&#8217;t handles.\n(You can also view today&#8217;s example as a generalization of the\n<A HREF=\"http:\/\/blogs.msdn.com\/oldnewthing\/archive\/2008\/08\/27\/8898863.aspx\">\nsentinel value problem<\/A>,\nwhere we need to generate an arbitrary number of sentinel values dynamically.\nActually, multiplexing <CODE>HANDLE<\/CODE> with <CODE>HRESULT<\/CODE>\nis also just another special case: We expanded the handle namespace\nto include error codes too.)\n<\/P>\n<P>\nAs I noted in the base article,\nthe people who are most interested in this sort of thing are people\nwriting low-level class libraries or wrapping kernel objects inside\na larger framework.\n<\/P>\n<P>\nSuppose, for example, you are writing a library that manipulates\nkernel objects, but you also have private types of objects (say,\na handle to a remote computer)\nthat you also want this library to be able to manipulate.\nOne way of doing this is to wrap everything inside some base class\nthat virtualizes away what type of handle you&#8217;re working on:\n<\/P>\n<PRE>\nclass ExtendedHandle {\npublic:\n  virtual ExtendedHandleType GetType() = 0;\n  virtual ~ExtendedHandle() { }\n};<\/p>\n<p>class KernelExtendedHandle : public ExtendedHandle {\npublic:\n  static KernelExtendedHandle *Create(&#8230;);\n  ExtendedHandleType GetType() { return KernelHandle; }\n  HANDLE GetHandle() { return Handle; }\nprivate:\n  KernelExtendedHandle(&#8230;);\n  HANDLE Handle;\n};<\/p>\n<p>class RemoteComputerExtendedHandle : public ExtendedHandle {\npublic:\n  static RemoteComputerExtendedHandle *Create(&#8230;);\n  ExtendedHandleType GetType() { return RemoteComputer; }\n  LPCTSTR GetComputerName() { &#8230; }\n  &#8230; other remote computer methods &#8230;\nprivate:\n  RemoteComputerExtendedHandle(&#8230;);\n  &#8230; stuff necessary for tracking remote computers &#8230;\n};\n<\/PRE>\n<P>\nNow, your library spends only 1% of its time manipulating these\nprivate object types;\nmost of the time, it&#8217;s dealing with regular kernel handles.\nIn other words,\n99% of your objects are of type <CODE>KernelExtendedHandle<\/CODE>.\nWhat used to be just a <CODE>HANDLE<\/CODE> (4&nbsp;bytes)\nis now a <CODE>EHANDLE<\/CODE> that in turn points to an 8-byte\nstructure\n(4&nbsp;bytes for the vtable and\n4&nbsp;bytes for the <CODE>HANDLE<\/CODE>).\nYour memory requirements have tripled <I>and<\/I> you added\nanother level of indirection (costing you locality),\njust for that 1% case.\n<\/P>\n<P>\nBut you can do better if you have those extra bits to play with.\nSince 99% of the objects you&#8217;re wrapping are just plain old kernel\nhandles, you can say that\nif the bottom bit is clear, then it&#8217;s just a kernel handle,\nand if the bottom bit is set, then the upper bits tell us what\nwe are really operating on.\n<\/P>\n<PRE>\ntypedef void *EHANDLE;<\/p>\n<p>BOOL IsKernelHandle(EHANDLE Handle)\n{\n  return (!((INT_PTR)Handle &amp; 1));\n}<\/p>\n<p>EHANDLE CreateHandleFromKernelHandle(HANDLE Handle)\n{\n  \/\/ if this assertion fires, then somebody tried to\n  \/\/ use an invalid kernel handle!\n  assert((!((INT_PTR)Handle &amp; 1));\n  return (EHANDLE)Handle;\n}<\/p>\n<p>EHANDLE CreateHandleFromExtendedHandle(ExtendedHandle Handle)\n{\n  \/\/ if this assertion fires, then somebody allocated\n  \/\/ a misaligned ExtendedHandle!\n  assert(!((INT_PTR)Handle &amp; 1));\n  return (EHANDLE)((INT_PTR)Handle | 1));\n}<\/p>\n<p>ExtendedHandle *GetExtendedHandle(EHANDLE Handle)\n{\n  assert(!IsKernelHandle(Handle));\n  return (ExtendedHandle*)((INT_PTR)Handle &amp; ~1);\n}<\/p>\n<p>ExtendedHandleType GetType(EHANDLE Handle)\n{\n if (IsKernelHandle(Handle)) {\n   return KernelHandleType;\n } else {\n   return GetExtendedHandle(Handle)-&gt;GetType();\n }\n}<\/p>\n<p>void ECloseHandle(EHANDLE Handle)\n{\n  if (IsKernelHandle(Handle))\n  {\n    CloseHandle(GetKernelHandle(Handle));\n  } else {\n   delete GetExtendedHandle(Handle);\n  }\n}\n<\/PRE>\n<P>\nNow the cost of a wrapped kernel handle is just 4&nbsp;bytes:\n4&nbsp;bytes for the <CODE>EHANDLE<\/CODE>, which also doubles\nas the actual kernel handle.\nThe cost of wrapped pseudo-handles is the same as before\n(4&nbsp;bytes for the <CODE>EHANDLE<\/CODE>, plus the size\nof the corresponding <CODE>XxxExtendedHandle<\/CODE> class).\nWe used the trick from last time in order to pack 33&nbsp;bits\ninto only 32&nbsp;bits:\nSince we know that the bottom bit of both kernel <CODE>HANDLE<\/CODE>s\nand <CODE>ExtendedHandle<\/CODE> pointers are zero,\nwe can use it as a discriminator.\n<\/P>\n<P>\nIf you are not confident that your <CODE>ExtendedHandle<\/CODE>\nclasses all use aligned pointers, you can use a different packing\nmechanism by using your own handle translation table:\n<\/P>\n<PRE>\nExtendedHandle *ExtendedHandleTable;<\/p>\n<p>ExtendedHandle *GetExtendedHandle(EHANDLE Handle)\n{\n  assert(!IsKernelHandle(Handle));\n  return ExtendedHandleTable[(INT_PTR)Handle &gt;&gt; 1];\n}\n<\/PRE>\n<P>\nUsing this technique, the upper 31&nbsp;bits of an <CODE>EHANDLE<\/CODE>\nwhich refers to an <CODE>ExtendedHandle<\/CODE> is an index into a\nprivately-managed table of <CODE>ExtendedHandle<\/CODE> objects.\n<\/P>\n<P>\nThis <I>secondary handle table<\/I> technique\nis entirely analogous to\n<A HREF=\"http:\/\/blogs.msdn.com\/oldnewthing\/archive\/2008\/02\/28\/7925962.aspx#7946255\">\nthe trick which one Posix library uses<\/A>\nto distinguish &#8220;real&#8221; process IDs from &#8220;virtual&#8221; process IDs,\nexcept that they are relying on undocumented behavior because\nthe bottom bits of process IDs are not guaranteed to be zero!\n<\/P>\n<P>\nSo there you have it, three scenarios where you can take advantage\nof knowing that\nthe bottom bits of kernel handles are always zero.\n<\/P><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Last time, we saw how those extra bits can be used to multiplex HANDLE with other values. That was a specific case of a more general scenario: Expanding the handle namespace to include things that aren&#8217;t handles. (You can also view today&#8217;s example as a generalization of the sentinel value problem, where we need to [&hellip;]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-21033","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"<p>Last time, we saw how those extra bits can be used to multiplex HANDLE with other values. That was a specific case of a more general scenario: Expanding the handle namespace to include things that aren&#8217;t handles. (You can also view today&#8217;s example as a generalization of the sentinel value problem, where we need to [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/21033","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=21033"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/21033\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=21033"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=21033"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=21033"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}