When you upgrade your resource strings to Unicode, don’t forget to specify the L prefix

Some time ago, I discussed how the Resource Compiler defaults to CP_ACP, even in the face of subtle hints that the file is UTF-8.

After yet another incident of Visual Studio secretly changing the file encoding from 1252 to UTF-8 and breaking all non-ASCII strings, combined with Azure DevOps and Visual Studio simply ignoring encoding changes when showing diffs, a colleague decided to solve the problem once and for all by using explicit Unicode escapes \x#### to represent non-ASCII characters. That way, it doesn’t matter whether the file encoding is 1252 or UTF-8 because the two code pages agree on the common ASCII subset.

What used to be

IDS_AWESOME "That’s great!"

was changed to

IDS_AWESOME "That\x2019s great!"

Unfortunately, the resulting string that appeared on screen was

That 19s great!

What went wrong?

If you are encoding Unicode into your string, you have to put an L prefix on the quoted string. Otherwise, the \xABCD sequence is interpreted as an 8-bit \xAB escape sequence, followed by two literal characters CD. In this case, the \x2019 was interpreted as \x20 (which encodes a space) followed by the literal characters 19, resulting in the string That␣19s great!.

The correct conversion includes the L prefix.

IDS_AWESOME L"That\x2019s great!"