{"id":16443,"date":"2009-10-08T10:00:00","date_gmt":"2009-10-08T10:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/2009\/10\/08\/what-is-the-format-of-a-double-null-terminated-string-with-no-strings\/"},"modified":"2009-10-08T10:00:00","modified_gmt":"2009-10-08T10:00:00","slug":"what-is-the-format-of-a-double-null-terminated-string-with-no-strings","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20091008-00\/?p=16443","title":{"rendered":"What is the format of a double-null-terminated string with no strings?"},"content":{"rendered":"<p>\nOne of the data formats peculiar to Windows is the\ndouble-null-terminated string.\nIf you have a bunch of strings and you want to build\none of these elusive double-null-terminated strings out of it,\nit&#8217;s no big deal.\n<\/p>\n<table style=\"font-family: monospace;text-align: center\" cellpadding=\"2\">\n<tbody>\n<tr>\n<td style=\"border: 0.75pt solid black;width: 2em\">H<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">e<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">l<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">l<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">o<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">\\0<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">w<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">o<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">r<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">l<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">d<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">\\0<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">\\0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>\nBut what about the edge cases?\nWhat if you want to build a double-null-terminated string\nwith no strings?\n<\/p>\n<p>\nLet&#8217;s step back and look at the double-null-terminated string\nwith two strings in it.\nBut I&#8217;m going to insert line breaks to highlight the structure.\n<\/p>\n<table style=\"font-family: monospace;text-align: center\" cellpadding=\"2\">\n<tbody>\n<tr>\n<td style=\"border: 0.75pt solid black;width: 2em\">H<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">e<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">l<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">l<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">o<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">\\0<\/td>\n<\/tr>\n<tr>\n<td style=\"border: 0.75pt solid black;width: 2em\">w<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">o<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">r<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">l<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">d<\/td>\n<td style=\"border: 0.75pt solid black;width: 2em\">\\0<\/td>\n<\/tr>\n<tr>\n<td style=\"border: 0.75pt solid black;width: 2em\">\\0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>\nNow I&#8217;m going to move the lines around.\n<\/p>\n<table style=\"font-family: monospace\" cellpadding=\"2\">\n<tbody>\n<tr>\n<td>\n<span style=\"border: 0.75pt solid black\">Hello\\0<\/span>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<span style=\"border: 0.75pt solid black\">world\\0<\/span>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<span style=\"border: 0.75pt solid black\">\\0<\/span>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>\nThis alternate way of writing the double-null-terminated string\nis the secret.\nInstead of viewing the string as something terminated by\ntwo consecutive null terminators, let&#8217;s view it as a\nlist of null-terminated strings, with a zero-length string at the end.\nAlternatively, think of it as a packed array of null-terminated\nstrings, with a zero-length string as the terminator.\n<\/p>\n<p>\nThis type of reinterpretation happens a lot in advanced\nmathematics.\nYou have some classical definition of an object,\nand then you invent a new interpretation which agrees\nwith the classical definition, but which gives you a\ndifferent perspective on the system and even generalizes\nto cases the classical definition didn&#8217;t handle.\n<\/p>\n<p>\nFor example, this &#8220;modern reinterpretation&#8221; of\ndouble-null-terminated strings provides another answer\nto a standard question:\n<\/p>\n<p>\nHow do I build a double-null-terminated string with an empty\nstring as one of the strings in the list?\n<\/p>\n<p>\nYou can&#8217;t, because the empty string is treated as the end of the\nlist.\nIt&#8217;s the same reason why you can&#8217;t put a null character\ninside a null-terminated string:\nThe null character is treated as the terminator.\nAnd in a double-null-terminated string,\nan empty string is treated as the terminator.\n<\/p>\n<table style=\"font-family: monospace\" cellpadding=\"2\">\n<tbody>\n<tr>\n<td>\n<span style=\"border: 0.75pt solid black\">One\\0<\/span>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<span style=\"border: 0.75pt solid black\">\\0<\/span>\n<\/td>\n<\/tr>\n<tr>\n<\/tr>\n<tr>\n<td>\n<span style=\"border: 0.75pt solid black\">Three\\0<\/span>\n<\/td>\n<\/tr>\n<tr>\n<td>\n<span style=\"border: 0.75pt solid black\">\\0<\/span>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>\nIf you try to put a zero-length string in your list,\nyou end up accidentally terminating it prematurely.\nUnder the classical view, you can see the two consecutive\nnull terminators:\nThey come immediately after the\n<code>\"One\"<\/code>.\nUnder the reinterpretation I propose,\nit&#8217;s more obvious,\nbecause the zero-length string is itself the terminator.\n<\/p>\n<p>\nIf you&#8217;re writing a helper class to manage double-null-terminated strings,\nmake sure you watch out for these empty strings.\n<\/p>\n<p>\nThis reinterpretation of a double-null-terminated string as really\na <i>list of strings with an empty string as the terminator<\/i>\nmakes writing code to walk through a double-null-terminated string\nquite straightforward:\n<\/p>\n<pre>for (LPTSTR pszz = pszzStart; *pszz; pszz += lstrlen(pszz) + 1) {<br \/> ... do something with pszz ...<br \/>}<br \/><\/pre>\n<p>\nDon&#8217;t think about looking for the double null terminator.\nInstead, just view it as a list of strings,\nand you stop when you find a string of length zero.\n<\/p>\n<p>\nThis reinterpretation also makes it clear how you express a list\nwith no strings in it at all:\nAll you have is the zero-length string terminator.\n<\/p>\n<table style=\"font-family: monospace;text-align: center\" cellpadding=\"2\">\n<tbody>\n<tr>\n<td style=\"border: 0.75pt solid black\">\\0<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>\nWhy do we even have double-null-terminated strings at all?\nWhy not just pass an array of pointers to strings?\n<\/p>\n<p>\nThat would have worked, too,\nbut it makes allocating and freeing the array more complicated,\nbecause the memory for the array and the component strings are\nnow scattered about.\n(Compare absolute and self-relative security descriptors.)\nA double-null-terminated string occupies a single block of memory\nwhich can be allocated and freed at one time,\nwhich is very convenient when you have to serialize and deserialize.\nIt also avoids questions like\n&#8220;Is it legal for two entries in the array to point to the same string?&#8221;\n<\/p>\n<p>\nKeeping it in a single block of memory reduces the number of selectors\nnecessary to represent the data in 16-bit Windows.\n(And this data representation was developed long before the 80386\nprocessor even existed.)\nAn array of pointers to 16 strings would require 17 selectors,\nif you used <code>GlobalAlloc<\/code> to allocate the memory:\none for the array itself,\nand one for each string.\nSelectors were a scarce resource in 16-bit Windows;\nthere were only 8192 of them available in the entire system.\nYou don&#8217;t want to use 1% of your system&#8217;s entire allocation\njust to represent an array of 100 strings.\n<\/p>\n<p>\nOne convenience of double-null-terminated strings is that you can load one\ndirectly out of\nyour resources with a single call to <code>LoadString<\/code>:\n<\/p>\n<pre>STRINGTABLE<br \/>BEGIN<br \/> IDS_FILE_FILTER \"Text files\\0*.txt\\0All files\\0*.*\\0\"<br \/>END<br \/><br \/>TCHAR szFilter[80];<br \/>LoadString(hinst, IDS_FILE_FILTER, szFilter, 80);<br \/><\/pre>\n<p>\nThis is very handy because it allows new filters to be added\nby simply changing a resource.\nIf the filter were passed as an array of pointers to strings,\nyou would probably put each string in a separate resource,\nand then the number of strings becomes more difficult to update.\n<\/p>\n<p>\nBut there is a gotcha in the above code,\nwhich we will look at next time.\n<\/p>\n<p>\n<b>Bonus Gotcha<\/b>:\nEven though you may know how double-null terminated strings work,\nthis doesn&#8217;t guarantee that the code you&#8217;re interfacing with\nunderstands it as well as you do.\nConsequently, you&#8217;d be best off putting the extra null terminator\nat the end if you are generating a double-null-terminated string,\njust in case the code you are calling expects the extra null\nterminator (even though it technically isn&#8217;t necessary).\nExample:\nThe ANSI version of\n<a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/ms682425.aspx\">\n<code>CreateProcess<\/code><\/a> locates the end of the environment block by\nlooking for two consecutive NULL bytes instead of looking for\nthe empty string terminator.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>One of the data formats peculiar to Windows is the double-null-terminated string. If you have a bunch of strings and you want to build one of these elusive double-null-terminated strings out of it, it&#8217;s no big deal. H e l l o \\0 w o r l d \\0 \\0 But what about the edge [&hellip;]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-16443","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"<p>One of the data formats peculiar to Windows is the double-null-terminated string. If you have a bunch of strings and you want to build one of these elusive double-null-terminated strings out of it, it&#8217;s no big deal. H e l l o \\0 w o r l d \\0 \\0 But what about the edge [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/16443","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=16443"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/16443\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=16443"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=16443"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=16443"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}