{"id":19483,"date":"2009-01-15T10:00:00","date_gmt":"2009-01-15T10:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/2009\/01\/15\/what-is-this-magic-setting-that-synthesizes-unicode-from-non-unicode\/"},"modified":"2009-01-15T10:00:00","modified_gmt":"2009-01-15T10:00:00","slug":"what-is-this-magic-setting-that-synthesizes-unicode-from-non-unicode","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20090115-00\/?p=19483","title":{"rendered":"What is this magic setting that synthesizes Unicode from non-Unicode?"},"content":{"rendered":"

Commenter dan g. wonders how Windows can treat non-Unicode applications as Unicode via the Regional and Language Options control panel<\/a>, specifically the part that lets you choose the Language for non-Unicode programs<\/i>. “Having always believed that the only way to display, say, Chinese characters correctly was to compile with _UNICODE, this facility seems all the more remarkable.”<\/p>\n

This setting is really not as magical as it appears. (After all, we had Chinese versions of 16-bit Windows that displayed Chinese characters just fine, and they certainly didn’t use Unicode since Unicode hadn’t been invented yet.) Michael Kaplan<\/a> went through this and many other settings in the Regional and Language Options control panel<\/a>, and from the chart at the top of the page, you see what Windows XP calls the Language for Non-Unicode Programs<\/i> used to go by the name Default System Locale<\/i>. The old name does a better job of describing what it actually does but does a worse job of describing what it’s used for<\/i>. <\/p>\n

In Win32, three character encodings have special status. Unicode (more precisely, UTF-16) of course is what Windows uses internally. There are also two 8-bit code pages<\/a>: CP_ACP<\/code>, the so-called ANSI code page<\/i> (even though it isn’t actually ANSI<\/a>), and the CP_OEM<\/code> code page, the so-called OEM code page<\/i> (even though it isn’t provided by the OEM<\/a>). <\/p>\n

When a non-Unicode program calls a function like TextOutA<\/code> to display a string represented in the ANSI code page, the string is converted to Unicode via the CP_ACP<\/code> code page. The Language for non-Unicode programs<\/i> setting controls what code page CP_ACP<\/code> corresponds to. On U.S. systems, it’s typically code page 1252<\/a>, but you can change it via that control panel. And that’s where it becomes possible to display Chinese characters without using Unicode. <\/p>\n

For example, code page 950<\/a> is a double-byte code page commonly seen in countries that use traditional Chinese characters. It can represent the English alphabet of A-Z, and through the use of double-byte characters can also represent a wide array of traditional Chinese characters, such as this block of characters which are represented by byte sequences of the form B3 40 through B3 FE<\/a>. If the ANSI code page is code page 950 and you pass data formatted for that code page to, say, the TextOutA<\/code> function, the corresponding Chinese characters will display, even though the program itself doesn’t use Unicode explicitly. <\/p>\n

That’s why it’s called the Language for non-Unicode programs<\/i>. It specifies which character set non-Unicode data should be interpreted as. <\/p>\n","protected":false},"excerpt":{"rendered":"

Commenter dan g. wonders how Windows can treat non-Unicode applications as Unicode via the Regional and Language Options control panel, specifically the part that lets you choose the Language for non-Unicode programs. “Having always believed that the only way to display, say, Chinese characters correctly was to compile with _UNICODE, this facility seems all the […]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[26],"class_list":["post-19483","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-other"],"acf":[],"blog_post_summary":"

Commenter dan g. wonders how Windows can treat non-Unicode applications as Unicode via the Regional and Language Options control panel, specifically the part that lets you choose the Language for non-Unicode programs. “Having always believed that the only way to display, say, Chinese characters correctly was to compile with _UNICODE, this facility seems all the […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/19483","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=19483"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/19483\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=19483"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=19483"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=19483"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}