{"id":253,"date":"2014-08-13T07:00:00","date_gmt":"2014-08-13T07:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/2014\/08\/13\/keep-your-eye-on-the-code-page-c-edition-the-mysterious-third-code-page\/"},"modified":"2014-08-13T07:00:00","modified_gmt":"2014-08-13T07:00:00","slug":"keep-your-eye-on-the-code-page-c-edition-the-mysterious-third-code-page","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20140813-00\/?p=253","title":{"rendered":"Keep your eye on the code page: C# edition (the mysterious third code page)"},"content":{"rendered":"<p>\nA customer was having trouble manipulating the console from a C#\nprogram:\n<\/p>\n<blockquote CLASS=\"q\">\n<p>\nWe found that C# can read only ASCII data from the console.\nIf we try to read non-ASCII data, we get garbage.\n<\/p>\n<pre>\nusing System;\nusing System.Text;\nusing System.Runtime.InteropServices;\nclass Program\n{\n  [StructLayout(LayoutKind.Sequential)]\n  struct COORD\n  {\n    public short X;\n    public short Y;\n  }\n  [DllImport(\"kernel32.dll\", SetLastError=true)]\n  static extern IntPtr GetStdHandle(int nStdHandle);\n  const int STD_OUTPUT_HANDLE = -11;\n  [DllImport(\"kernel32.dll\", SetLastError=true)]\n  static extern bool ReadConsoleOutputCharacter(\n    IntPtr hConsoleOutput,\n    [Out] StringBuilder lpCharacter,\n    uint nLength,\n    COORD dwReadCoord,\n    out uint lpNumberOfCharsRead);\n  public static void Main()\n  {\n    \/\/ Write a string to a fixed position\n    System.Console.Clear();\n    System.Console.WriteLine(\"\\u00C5ngstr\\u00f6m\");\n    \/\/ Read it back\n    COORD coord  = new COORD { X = 0, Y = 0 };\n    StringBuilder sb = new StringBuilder(8);\n    uint nRead = 0;\n    ReadConsoleOutputCharacter(GetStdHandle(STD_OUTPUT_HANDLE),\n                               sb, (uint)sb.Capacity, coord, out nRead);\n    \/\/ Trim off any unused excess.\n    sb.Remove((int)nRead, sb.Length - (int)nRead);\n    \/\/ Show what we read\n    System.Console.WriteLine(sb);\n  }\n}\n<\/pre>\n<p>\nObserve that this program is unable to read the\n&Aring; and &ouml; characters.\nThey come back as garbage.\n<\/p>\n<\/blockquote>\n<p>\nAlthough there are three code pages that have special treatment in Windows,\n<a HREF=\"http:\/\/msdn.microsoft.com\/library\/7b93s42f.aspx\">\nthe CLR gives access to only two of them via <code>Dll&shy;Import<\/code><\/a>.\n<\/p>\n<ul>\n<li>CharSet.Ansi = CP_ACP\n<li>CharSet.Unicode = Unicode (which in Windows means UTF16-LE\n    unless otherwise indicated).\n<\/ul>\n<p>\nUnfortunately,\nthe console traditionally uses the OEM code page.\n<\/p>\n<p>\nSince the <code>Dll&shy;Import<\/code> did not specify a character set,\nthe CLR defaults\n(<a HREF=\"http:\/\/blogs.msdn.com\/b\/oldnewthing\/archive\/2014\/08\/12\/10549258.aspx\">unfortunately<\/a>)\nto <code>Char&shy;Set.Ansi<\/code>.\nResult:\nThe\n<code>Read&shy;Console&shy;Output&shy;Character<\/code> function stores\nits results in <code>CP_OEM<\/code>,\nthe CLR treats the buffer as if it were <code>CP_ACP<\/code>,\nand the result is confusion.\n<\/p>\n<p>\nThe narrow-minded fix is to try to fix the\n<a HREF=\"http:\/\/en.wikipedia.org\/wiki\/Mojibake\">\nmojibake<\/a>\nby taking the misconverted Unicode string,\nconverting it to bytes via the ANSI code page,\nthen converting the bytes to Unicode via the OEM code page.\n<\/p>\n<p>\nThe better fix is simply to avoid the 8-bit code page issues entirely\nand say you want to use Unicode.\n<\/p>\n<pre>\n  [DllImport(\"kernel32.dll\", SetLastError=true, <font COLOR=\"blue\">CharSet=CharSet.Unicode<\/font>)]\n  static extern bool ReadConsoleOutputCharacter(...);\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>A customer was having trouble manipulating the console from a C# program: We found that C# can read only ASCII data from the console. If we try to read non-ASCII data, we get garbage. using System; using System.Text; using System.Runtime.InteropServices; class Program { [StructLayout(LayoutKind.Sequential)] struct COORD { public short X; public short Y; } [DllImport(&#8220;kernel32.dll&#8221;, [&hellip;]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-253","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"<p>A customer was having trouble manipulating the console from a C# program: We found that C# can read only ASCII data from the console. If we try to read non-ASCII data, we get garbage. using System; using System.Text; using System.Runtime.InteropServices; class Program { [StructLayout(LayoutKind.Sequential)] struct COORD { public short X; public short Y; } [DllImport(&#8220;kernel32.dll&#8221;, [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/253","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=253"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/253\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=253"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=253"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=253"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}