{"id":109915,"date":"2024-06-19T07:00:00","date_gmt":"2024-06-19T14:00:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/oldnewthing\/?p=109915"},"modified":"2024-06-19T14:17:46","modified_gmt":"2024-06-19T21:17:46","slug":"20240619-00","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20240619-00\/?p=109915","title":{"rendered":"On the sadness of treating counted strings as null-terminated strings"},"content":{"rendered":"

There are a number of data types which represent a counted string. Some of them are in the C++ standard library, like std::string<\/code> and std::wstring<\/code>. Some of them are Windows-specific like BSTR<\/code> or HSTRING<\/code>. Be careful when treating these counted strings as null-terminated strings.<\/p>\n

Treating a counted string as a null-terminated string is a lossy operation, because any embedded nulls in the counted string are mistakenly interpreted as the end of the string.<\/p>\n

std::string s = \"hello\\0world\"s;\r\n\r\n\/\/ This prints \"hello<nul>world\"\r\nstd::cout << s << std::endl;\r\n\r\n\/\/ Copy it through c_str\r\nstd::string t = s.c_str();\r\n\r\n\/\/ This prints \"hello\"\r\nstd::cout << t << std::endl;\r\n<\/pre>\n
The embedded null in the string s<\/code> is treated as the string terminator when we interpret the c_str()<\/code> as a null-terminated string, and the last part of the string is lost.<\/p>\n
Now, you wouldn’t be so silly as to copy a std::string<\/code> that way, seeing as there is a copy constructor right there.<\/p>\n
std::string t = s; \/\/ use the copy constructor\r\n<\/pre>\n
But when you’re converting between different counted string types, you may be tempted to use the null-terminated string as the intermediary.<\/p>\n
\/\/ widget.GetName() returns a winrt::hstring,\r\n\/\/ but we want to manipulate it as a std::wstring\r\nstd::wstring name(widget.GetName().c_str());\r\n<\/pre>\n
Not only is there a performance penalty here, because the std::wstring<\/code> constructor has to go look for the terminating null character, but there is also a security vulnerability: If an attacker puts an embedded null in a string, they might be able to sneak past a security check or validation.<\/p>\n
bool IsAllowedName(std::wstring const& name)\r\n{\r\n    return name == L\"alice\" || name == L\"bob\";\r\n}\r\n\r\nvoid ProcessWidget(Widget const& widget)\r\n{\r\n    if (!IsAllowedName(widget.Name().c_str())) {\r\n        throw winrt::hresult_access_denied();\r\n    }\r\n\r\n    \u27e6 continue processing \u27e7\r\n}\r\n<\/pre>\n
An attacker could bypass the access check by using a widget whose name is \"alice\\0haha\"<\/code>, and it will be considered to have an allowed name, since the embedded null causes the std::wstring<\/code> passed to Is\u00adAllowed\u00adName()<\/code> to consist only of the characters leading up to the null terminator.<\/p>\n
As another example, you might want to print a BSTR<\/code>, which is also a counted string type, although the representation is that of a pointer to the first wchar_t<\/code>. This means that you can often pretend that a BSTR<\/code> is a null-terminated string, but the danger is that any embedded null will cause you to stop processing the string before you get to the end.<\/p>\n
void PrintBstr(BSTR bstr)\r\n{\r\n    std::cout << bstr;\r\n}\r\n<\/pre>\n
Sidebar<\/b>: There’s another danger, namely that the BSTR<\/code> might be nullptr<\/code>, which represents a zero-length string. However, trying to <<<\/code> a (wchar_t*)nullptr<\/code> will crash because the <<<\/code> operator will dereference the null pointer while searching for the null terminator.<\/p>\n
Okay, now that we’ve laid out the problem, we’ll look at solutions next time.<\/p>\n
Bonus chatter<\/b>:<\/p>\n
\/\/ This also prints \"hello\"\r\nstd::string u = \"hello\\0world\";\r\nstd::cout << u << std::endl;\r\n<\/pre>\n
Bonus bonus chatter<\/b>: For the specific case of converting a winrt::hstring<\/code> to a std::wstring<\/code>, you can just pass the winrt::hstring<\/code> and use the std::wstring_view<\/code> conversion constructor!<\/p>\n
winrt::hstring h;\r\nstd::wstring w(h); \/\/ just construct it from the hstring\r\n<\/pre>\n
In our example, we would just pass the winrt::hstring<\/code> to Is\u00adAllowed\u00adName<\/code> and let the compiler do the conversion.<\/p>\n
void ProcessWidget(Widget const& widget)\r\n{\r\n    if (!IsAllowedName(widget.Name())) {\r\n        throw winrt::hresult_access_denied();\r\n    }\r\n\r\n    \u27e6 continue processing \u27e7\r\n}\r\n<\/pre>\n
Bonus bonus bonus chatter<\/b>: What if you are forced to produce a pointer to a null-terminated string due to some interop requirement?<\/p>\n
In that case, you should fail the operation if the counted string contains an embedded null. For HSTRING<\/code>, you can use the Windows\u00adString\u00adHas\u00adEmbedded\u00adNull<\/code> to check for an embedded null. The Windows\u00adString\u00adHas\u00adEmbedded\u00adNull<\/code> function caches the result, so asking a second time uses the result calculated from the first time. Mind you, scanning a string for an embedded null is probably not that expensive, so the cache doesn’t buy you much, especially since you’re probably about to pass the string to another function that will consume it, so the contents of the string are going to be scanned by the consumer anyway.<\/p>\n
Arguably, the c_str()<\/code> function should throw an exception if the counted string is not representable as a C-style null-terminated string. But what’s done is done. At best, we can make up a new method name, like safe_c_str()<\/code>?<\/p>\n","protected":false},"excerpt":{"rendered":"
You’re throwing away perfectly good data, there.<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-109915","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"
You’re throwing away perfectly good data, there.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/109915","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=109915"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/109915\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=109915"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=109915"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=109915"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}