\nToday’s Little Program strips diacritics from a Unicode string.\nWhy?\nHey, I said that Little Programs require little to no motivation.\nIt might come in handy in a spam filter, since it was popular,\nat least for a time, to put random accent marks on spam subject\nlines in order to sneak past keyword filters.\n(It doesn’t seem to be popular any more.)\n<\/p>\n
\nThis is basically a C-ization of\n\nthe C# code originally written by Michael Kaplan<\/a>.\nDon’t forget to read\n\nthe follow-up discussion that notes that this can result in strange\nresults<\/a>.\n<\/p>\n \nFirst, let’s create our dialog box.\nNote that I intentionally give it a huge font\nso that the diacritics are easier to see.\n<\/p>\n \nNow the program that uses the dialog box.\n<\/p>\n \nThe \nOkay, here’s where the actual work happens.\nWe put the source string into Normalization Form KD.\nThis decomposes the diacritics so that we can identify them\nwith \nOf course, in real life, you wouldn’t hard-code the array sizes\nlike I did here, but this is just a Little Program,\nand Little Programs are allowed to take shortcuts.\n<\/p>\n \nThe rest of the program is just a framework to get into that\nfunction.\n<\/p>\n \nOkay, let’s take this program for a spin.\nHere are some interesting characters to try:\n<\/p>\n \nThere are some interesting quirks here.\nMind you, this is what the Unicode Consortium says,\nso if you think they are wrong,\nyou can take it up with them.\n<\/p>\n \nThe superscript-like characters are converted to their plain\nversions.\nEnclosed alphabetics are also converted,\nbut not the ® symbol.\nFullwidth forms of Latin letters\nare converted to their halfwidth equivalents.\nOn the other hand, halfwidth Katakana characters are expanded to their\nfullwidth equivalents.\nBut small Katakana does not convert to their large equivalents.\n<\/p>\n \nThe Ø disappears completely! What’s up with that?\nThe character code for Ø is reported as\n \nThe diacritic removal ignores linguistic rules.\nThe Swedish Å decomposes into a capital A and a combining\nring above,\neven though in Swedish, the character is considered nondecomposable.\n(Just like the capital letter Q in English\ndoes not decompose into an O and a tail.)\nKatakana Go suffers a similar ignoble fate,\n\nconverting to Katakana Ko<\/a>,\nwhich is linguistically nonsensical.\nBut then again, removing diacritics\nis already linguistically nonsensical<\/i>.\nNonsensical operation is nonsensical.\n<\/p>\n \nThere is no attempt to unify look-alike characters from different\nscripts.\nLook-alike characters in the Greek and Cyrillic alphabets\nare not mapped to their Latin doppelgängers.\n<\/p>\n \nThe infamous Turkish dotless i does not turn into a dotted i.\n(And the lowercase Latin i does not decompose into a combining dot and a dotless i.)\n<\/p>\n \nFinally, I tried a selection of punctuation marks.\nMost of them pass through unchanged,\nwith the exception of the double prime and double exclamation mark\nwhich each decompose into a pair of singles.\n(But double quotation marks do not decompose into a pair of singles.)\n<\/p>\n \nOkay, but the goal of this exercise was spam detection,\nso we are actually interested in mapping as far as possible\nall the way down to plain ASCII.\nWe’d like to convert, for example,\nthe look-alike characters in the Cyrillic and Greek alphabets\nto the Latin characters they resemble.\n<\/p>\n \nSo let’s try something else.\nIf we want to convert to ASCII,\nthen just convert to ASCII!\n<\/p>\n \nWe can extend the table above with a new column.\n<\/p>\n \nThere are some interesting differences here.\n<\/p>\n \nSome characters fail to convert to ASCII outright.\nThis is not unexpected for the Japanese characters,\nis mildly unexpected for the\nlook-alikes in the Cyrillic and Greek alphabets,\nand is surprising for some characters like double prime,\ndouble exclamation point,\nenclosed alphanumerics,\nand vulgar fractions\nbecause they had ASCII decompositions in Normalization Form KD,\nbut converting directly into ASCII refused to use them.\n<\/p>\n \nBut the dotless i gets its dot back.\n<\/p>\n \nAnother weird thing you might notice is that the æ converts\nto just the a.\nThis goes contrary to the expectations of\nAmerican English,\nbecause words which historically use the æ and œ\nare largely\nrespelled in American English to use just the e.\n(Encyclopædia → encyclopedia,\nfœtus → fetus.)\nMysteries abound.\n<\/p>\n \nIf your real goal is to map every character to its nearest ASCII\nlook-alike,\nthen all these code page games are just beating around the bush.\nThe way to go is to use the Unicode Confusables database.\nThere is a\n\nhuge data file<\/a>\nand\n\ninstructions on how to use it<\/a>.\nThere’s also\n\na nice Web site<\/a>\nthat lets you explore the confusables database interactively.\n<\/p>\n \nOr you could just take the sledgehammer approach:\nIf there are a significant number of characters outside the Latin alphabet\nand punctuation and you are expecting English text,\nthen just reject it as likely spam.\n<\/p>\n \nಠ_ಠ<\/p>\n","protected":false},"excerpt":{"rendered":" Today’s Little Program strips diacritics from a Unicode string. Why? Hey, I said that Little Programs require little to no motivation. It might come in handy in a spam filter, since it was popular, at least for a time, to put random accent marks on spam subject lines in order to sneak past keyword filters. […]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-43553","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":" Today’s Little Program strips diacritics from a Unicode string. Why? Hey, I said that Little Programs require little to no motivation. It might come in handy in a spam filter, since it was popular, at least for a time, to put random accent marks on spam subject lines in order to sneak past keyword filters. […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/43553","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=43553"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/43553\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=43553"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=43553"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=43553"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n\/\/ scratch.h\n#define IDD_SCRATCH 1\n#define IDC_SOURCE 100\n#define IDC_SOURCEPOINTS 101\n#define IDC_DEST 102\n#define IDC_DESTPOINTS 103\n\/\/ scratch.rc\n#include <windows.h>\n#include \"scratch.h\"\nIDD_SCRATCH DIALOGEX 0, 0, 320, 88\nSTYLE DS_MODALFRAME | WS_POPUP | WS_CAPTION | WS_SYSMENU\nCaption \"Stripping diacritics\"\nFONT 20, \"MS Shell Dlg\"\nBEGIN\n LTEXT \"Original:\", -1, 4, 8, 38, 10\n EDITTEXT IDC_SOURCE, 46, 6, 270, 12, ES_AUTOHSCROLL\n LTEXT \"\", IDC_SOURCEPOINTS, 46, 22, 270, 12\n LTEXT \"Modified:\", -1, 4, 40, 38, 10\n EDITTEXT IDC_DEST, 46, 38, 270, 12, ES_AUTOHSCROLL\n LTEXT \"\", IDC_DESTPOINTS, 46, 54, 270, 12\n DEFPUSHBUTTON \"OK\", IDOK, 266, 70, 50, 14\nEND\n<\/pre>\n
\n\/\/ scratch.cpp\n#define STRICT\n#define UNICODE\n#define _UNICODE\n#include <windows.h>\n#include <windowsx.h>\n#include <strsafe.h>\n#include \"scratch.h\"\n#define MAXSOURCE 64\nvoid SetDlgItemCodePoints(HWND hwnd, int idc, PCWSTR psz)\n{\n wchar_t szResult[MAXSOURCE * 4 * 5];\n szResult[0] = 0;\n PWSTR pszResult = szResult;\n size_t cchResult = ARRAYSIZE(szResult);\n HRESULT hr = S_OK;\n for (; SUCCEEDED(hr) && *psz; psz++) {\n wchar_t szPoint[6];\n hr = StringCchPrintf(szPoint, ARRAYSIZE(szPoint), L\"%04x \", *psz);\n if (SUCCEEDED(hr)) {\n hr = StringCchCatEx(pszResult, cchResult, szPoint, &pszResult, &cchResult, 0);\n }\n }\n SetDlgItemText(hwnd, idc, szResult);\n}\n<\/pre>\nSetDlgItemCodePoints<\/code>\nfunction takes a UTF-16 string and prints all the code points.\nThis is just to help visualize the result;\nit’s not part of the actual diacritic-removal algorithm.\n<\/p>\n\nvoid OnUpdate(HWND hwnd)\n{\n wchar_t szSource[MAXSOURCE];\n GetDlgItemText(hwnd, IDC_SOURCE, szSource, ARRAYSIZE(szSource));\n wchar_t szDest[MAXSOURCE * 4];\n int cchActual = NormalizeString(NormalizationKD,\n szSource, -1,\n szDest, ARRAYSIZE(szDest));\n if (cchActual <= 0) szDest[0] = 0;\n WORD rgType[ARRAYSIZE(szDest)];\n GetStringTypeW(CT_CTYPE3, szDest, -1, rgType);\n PWSTR pszWrite = szDest;\n for (int i = 0; szDest[i]; i++) {\n if (!(rgType[i] & C3_NONSPACING)) {\n *pszWrite++ = szDest[i];\n }\n }\n *pszWrite = 0;\n SetDlgItemText(hwnd, IDC_DEST, szDest);\n SetDlgItemCodePoints(hwnd, IDC_SOURCEPOINTS, szSource);\n SetDlgItemCodePoints(hwnd, IDC_DESTPOINTS, szDest);\n}\n<\/pre>\nGetStringTypeW<\/code>\nand then strip them out.\n<\/p>\n\nINT_PTR CALLBACK DlgProc(HWND hwnd, UINT wm,\n WPARAM wParam, LPARAM lParam)\n{\n switch (wm)\n {\n case WM_INITDIALOG:\n return TRUE;\n case WM_COMMAND:\n switch (GET_WM_COMMAND_ID(wParam, lParam)) {\n case IDC_SOURCE:\n switch (GET_WM_COMMAND_CMD(wParam, lParam)) {\n case EN_UPDATE:\n OnUpdate(hwnd);\n break;\n }\n break;\n case IDOK:\n EndDialog(hwnd, 0);\n return TRUE;\n }\n break;\n case WM_CLOSE:\n EndDialog(hwnd, 0);\n return TRUE;\n }\n return FALSE;\n}\nint WINAPI wWinMain(HINSTANCE hinst, HINSTANCE hinstPrev,\n LPWSTR lpCmdLine, int nShowCmd)\n{\n DialogBox(hinst, MAKEINTRESOURCE(IDD_SCRATCH), nullptr, DlgProc);\n return 0;\n}\n<\/pre>\n\n
\n Original character<\/th>\n Resulting character<\/th>\n<\/tr>\n \n ª<\/td>\n 00AA<\/td>\n Feminine ordinal indicator<\/td>\n a<\/td>\n 0061<\/td>\n Latin small letter a<\/td>\n<\/tr>\n \n ¹<\/td>\n 00B1<\/td>\n Superscript one<\/td>\n 1<\/td>\n 0031<\/td>\n Digit one<\/td>\n<\/tr>\n \n ½<\/td>\n 00BD<\/td>\n Vulgar fraction one half<\/td>\n 1⁄2<\/td>\n 0031 2044 0032<\/td>\n Digit one + Fraction slash + Digit two<\/td>\n<\/tr>\n \n ı<\/td>\n 0131<\/td>\n Latin small letter dotless i<\/td>\n ı<\/td>\n 0131<\/td>\n Latin small letter dotless i<\/td>\n<\/tr>\n \n Ø<\/td>\n 00D8<\/td>\n Latin capital letter O with stroke<\/td>\n <\/td>\n <\/td>\n Disappears!<\/td>\n<\/tr>\n \n ł<\/td>\n 0142<\/td>\n Latin small letter l with stroke<\/td>\n ł<\/td>\n 0142<\/td>\n Latin small letter l with stroke<\/td>\n<\/tr>\n \n ŀ<\/td>\n 0140<\/td>\n Latin small letter l with middle dot<\/td>\n l·<\/td>\n 006C 00B7<\/td>\n Latin small letter l + middle dot<\/td>\n<\/tr>\n \n æ<\/td>\n 00E6<\/td>\n Latin small letter ae<\/td>\n æ<\/td>\n 00E6<\/td>\n Latin small letter ae<\/td>\n<\/tr>\n \n Ή<\/td>\n 0389<\/td>\n Greek capital letter Eta with tonos<\/td>\n Η<\/td>\n 0397<\/td>\n Greek capital letter Eta<\/td>\n<\/tr>\n \n А<\/td>\n 0410<\/td>\n Cyrillic capital letter А<\/td>\n А<\/td>\n 0410<\/td>\n Cyrillic capital letter А<\/td>\n<\/tr>\n \n Å<\/td>\n 00C5<\/td>\n Latin capital letter A with ring above<\/td>\n A<\/td>\n 0041<\/td>\n Latin capital letter A<\/td>\n<\/tr>\n \n Ａ<\/td>\n FF21<\/td>\n Fullwidth Latin capital letter A<\/td>\n A<\/td>\n 0041<\/td>\n Latin capital letter A<\/td>\n<\/tr>\n \n ①<\/td>\n 2460<\/td>\n Circled digit one<\/td>\n 1<\/td>\n 0031<\/td>\n Digit one<\/td>\n<\/tr>\n \n ➀<\/td>\n 2780<\/td>\n Dingbat circled sans-serif digit one<\/td>\n ➀<\/td>\n 2780<\/td>\n Dingbat circled sans-serif digit one<\/td>\n<\/tr>\n \n ®<\/td>\n 00AE<\/td>\n Registered sign<\/td>\n ®<\/td>\n 00AE<\/td>\n Registered sign<\/td>\n<\/tr>\n \n Ⓡ<\/td>\n 24c7<\/td>\n Circled Latin capital letter R<\/td>\n R<\/td>\n 0052<\/td>\n Latin capital letter R<\/td>\n<\/tr>\n \n 𝖕<\/td>\n D835 DD95<\/td>\n Mathematical bold Fraktur small p<\/td>\n p<\/td>\n 0070<\/td>\n Latin small letter p<\/td>\n<\/tr>\n \n ｬ<\/td>\n FF6C<\/td>\n Halfwidth Katakana letter small Ya<\/td>\n ャ<\/td>\n 30E3<\/td>\n Katakana letter small Ya<\/td>\n<\/tr>\n \n ャ<\/td>\n 30E3<\/td>\n Katakana letter small Ya<\/td>\n ャ<\/td>\n 30E3<\/td>\n Katakana letter small Ya<\/td>\n<\/tr>\n \n ゴ<\/td>\n 30B4<\/td>\n Katakana letter Go<\/td>\n コ<\/td>\n 30B3<\/td>\n Katakana letter Ko<\/td>\n<\/tr>\n \n “<\/td>\n 201C<\/td>\n Left double quotation mark<\/td>\n “<\/td>\n 201C<\/td>\n Left double quotation mark<\/td>\n<\/tr>\n \n ”<\/td>\n 201D<\/td>\n Right double quotation mark<\/td>\n ”<\/td>\n 201D<\/td>\n Right double quotation mark<\/td>\n<\/tr>\n \n „<\/td>\n 201E<\/td>\n Double low-9 quotation mark<\/td>\n „<\/td>\n 201E<\/td>\n Double low-9 quotation mark<\/td>\n<\/tr>\n \n ‟<\/td>\n 201F<\/td>\n Double high-reversed-9 quotation mark<\/td>\n ‟<\/td>\n 201F<\/td>\n Double high-reversed-9 quotation mark<\/td>\n<\/tr>\n \n ″<\/td>\n 2033<\/td>\n Double prime<\/td>\n ′′<\/td>\n 2032 2032<\/td>\n Prime + Prime<\/td>\n<\/tr>\n \n ‵<\/td>\n 2035<\/td>\n Reverse prime<\/td>\n ‵<\/td>\n 2035<\/td>\n Reverse prime<\/td>\n<\/tr>\n \n ‹<\/td>\n 2039<\/td>\n Single left-pointing angle quotation mark<\/td>\n ‹<\/td>\n 2039<\/td>\n Single left-pointing angle quotation mark<\/td>\n<\/tr>\n \n «<\/td>\n 00AB<\/td>\n Left-pointing double angle quotation mark<\/td>\n «<\/td>\n 00AB<\/td>\n Left-pointing double angle quotation mark<\/td>\n<\/tr>\n \n —<\/td>\n 2014<\/td>\n Em-dash<\/td>\n —<\/td>\n 2014<\/td>\n Em-dash<\/td>\n<\/tr>\n ‼<\/td>\n 203C<\/td>\n Double exclamation mark<\/td>\n !!<\/td>\n 0021 0021<\/td>\n Exclamation mark + Exclamation mark<\/td>\n<\/tr>\n<\/table>\n C3_ALPHA | C3_NONSPACING | C3_DIACRITIC<\/code>,\nand since we are removing nonspacing characters,\nthis causes it to be removed.\n(Why is Ø nonspacing? It occupies space!)\nFor whatever reason, it does not decompose into\nO + Combining Solidus Overlay.\nOn the other hand, the Polish ł remains intact\nbecause it is reported as\nC3_ALPHA | C3_DIACRITIC<\/code>.\nPoland wins and Norway loses?\n<\/p>\n\n#define CP_ASCII 20127\nvoid OnUpdate(HWND hwnd)\n{\n wchar_t szSource[MAXSOURCE];\n GetDlgItemText(hwnd, IDC_SOURCE, szSource, ARRAYSIZE(szSource));\n char szDest[MAXSOURCE * 2];\n int cchActual = WideCharToMultiByte(CP_ASCII, 0, szSource, -1,\n szDest, ARRAYSIZE(szDest), 0, 0);\n if (cchActual <= 0) szDest[0] = 0;\n SetDlgItemTextA(hwnd, IDC_DEST, szDest);\n SetDlgItemCodePoints(hwnd, IDC_SOURCEPOINTS, szSource);\n}\n<\/pre>\n\n
\n Original character<\/th>\n KD character<\/th>\n ASCII character<\/th>\n<\/tr>\n \n ª<\/td>\n 00AA<\/td>\n Feminine ordinal indicator<\/td>\n a<\/td>\n 0061<\/td>\n Latin small letter a<\/td>\n a<\/td>\n 0061<\/td>\n Latin small letter a<\/td>\n<\/tr>\n \n ¹<\/td>\n 00B1<\/td>\n Superscript one<\/td>\n 1<\/td>\n 0031<\/td>\n Digit one<\/td>\n 1<\/td>\n 0031<\/td>\n Digit one<\/td>\n<\/tr>\n \n ½<\/td>\n 00BD<\/td>\n Vulgar fraction one half<\/td>\n 1⁄2<\/td>\n 0031 2044 0032<\/td>\n Digit one + Fraction slash + Digit two<\/td>\n ?<\/td>\n <\/td>\n No conversion<\/td>\n<\/tr>\n \n ı<\/td>\n 0131<\/td>\n Latin small letter dotless i<\/td>\n ı<\/td>\n 0131<\/td>\n Latin small letter dotless i<\/td>\n i<\/td>\n 0069<\/td>\n Latin small letter i<\/td>\n<\/tr>\n \n Ø<\/td>\n 00D8<\/td>\n Latin capital letter O with stroke<\/td>\n <\/td>\n <\/td>\n Disappears!<\/td>\n O<\/td>\n 004F<\/td>\n Latin capital letter O<\/td>\n<\/tr>\n \n ł<\/td>\n 0142<\/td>\n Latin small letter l with stroke<\/td>\n ł<\/td>\n 0142<\/td>\n Latin small letter l with stroke<\/td>\n l<\/td>\n 006C<\/td>\n Latin small letter l<\/td>\n<\/tr>\n \n ŀ<\/td>\n 0140<\/td>\n Latin small letter l with middle dot<\/td>\n l·<\/td>\n 006C 00B7<\/td>\n Latin small letter l + middle dot<\/td>\n ?<\/td>\n <\/td>\n No conversion<\/td>\n<\/tr>\n \n æ<\/td>\n 00E6<\/td>\n Latin small letter ae<\/td>\n æ<\/td>\n 00E6<\/td>\n Latin small letter ae<\/td>\n a<\/td>\n 0061<\/td>\n Latin small letter a<\/td>\n<\/tr>\n \n Ή<\/td>\n 0389<\/td>\n Greek capital letter Eta with tonos<\/td>\n Η<\/td>\n 0397<\/td>\n Greek capital letter Eta<\/td>\n ?<\/td>\n <\/td>\n No conversion<\/td>\n<\/tr>\n \n А<\/td>\n 0410<\/td>\n Cyrillic capital letter А<\/td>\n А<\/td>\n 0410<\/td>\n Cyrillic capital letter А<\/td>\n ?<\/td>\n <\/td>\n No conversion<\/td>\n<\/tr>\n \n Å<\/td>\n 00C5<\/td>\n Latin capital letter A with ring above<\/td>\n A<\/td>\n 0041<\/td>\n Latin capital letter A<\/td>\n A<\/td>\n 0041<\/td>\n Latin capital letter A<\/td>\n<\/tr>\n \n Ａ<\/td>\n FF21<\/td>\n Fullwidth Latin capital letter A<\/td>\n A<\/td>\n 0041<\/td>\n Latin capital letter A<\/td>\n A<\/td>\n 0041<\/td>\n Latin capital letter A<\/td>\n<\/tr>\n \n ①<\/td>\n 2460<\/td>\n Circled digit one<\/td>\n 1<\/td>\n 0031<\/td>\n Digit one<\/td>\n ?<\/td>\n <\/td>\n No conversion<\/td>\n<\/tr>\n \n ➀<\/td>\n 2780<\/td>\n Dingbat circled sans-serif digit one<\/td>\n ➀<\/td>\n 2780<\/td>\n Dingbat circled sans-serif digit one<\/td>\n ?<\/td>\n <\/td>\n No conversion<\/td>\n<\/tr>\n \n ®<\/td>\n 00AE<\/td>\n Registered sign<\/td>\n ®<\/td>\n 00AE<\/td>\n Registered sign<\/td>\n R<\/td>\n 0052<\/td>\n Latin capital letter R<\/td>\n<\/tr>\n \n Ⓡ<\/td>\n 24c7<\/td>\n Circled Latin capital letter R<\/td>\n R<\/td>\n 0052<\/td>\n Latin capital letter R<\/td>\n ?<\/td>\n <\/td>\n No conversion<\/td>\n<\/tr>\n \n 𝖕<\/td>\n D835 DD95<\/td>\n Mathematical bold Fraktur small p<\/td>\n p<\/td>\n 0070<\/td>\n Latin small letter p<\/td>\n ??<\/td>\n <\/td>\n No conversion<\/td>\n<\/tr>\n \n ｬ<\/td>\n FF6C<\/td>\n Halfwidth Katakana letter small Ya<\/td>\n ャ<\/td>\n 30E3<\/td>\n Katakana letter small Ya<\/td>\n ?<\/td>\n <\/td>\n No conversion<\/td>\n<\/tr>\n \n ャ<\/td>\n 30E3<\/td>\n Katakana letter small Ya<\/td>\n ャ<\/td>\n 30E3<\/td>\n Katakana letter small Ya<\/td>\n ?<\/td>\n <\/td>\n No conversion<\/td>\n<\/tr>\n \n ゴ<\/td>\n 30B4<\/td>\n Katakana letter Go<\/td>\n コ<\/td>\n 30B3<\/td>\n Katakana letter Ko<\/td>\n ?<\/td>\n <\/td>\n No conversion<\/td>\n<\/tr>\n \n “<\/td>\n 201C<\/td>\n Left double quotation mark<\/td>\n “<\/td>\n 201C<\/td>\n Left double quotation mark<\/td>\n “<\/td>\n 0022<\/td>\n Quotation mark<\/td>\n<\/tr>\n \n ”<\/td>\n 201D<\/td>\n Right double quotation mark<\/td>\n ”<\/td>\n 201D<\/td>\n Right double quotation mark<\/td>\n “<\/td>\n 0022<\/td>\n Quotation mark<\/td>\n<\/tr>\n \n „<\/td>\n 201E<\/td>\n Double low-9 quotation mark<\/td>\n „<\/td>\n 201E<\/td>\n Double low-9 quotation mark<\/td>\n “<\/td>\n 0022<\/td>\n Quotation mark<\/td>\n<\/tr>\n \n ‟<\/td>\n 201F<\/td>\n Double high-reversed-9 quotation mark<\/td>\n ‟<\/td>\n 201F<\/td>\n Double high-reversed-9 quotation mark<\/td>\n ?<\/td>\n <\/td>\n No conversion<\/td>\n<\/tr>\n \n ″<\/td>\n 2033<\/td>\n Double prime<\/td>\n ′′<\/td>\n 2032 2032<\/td>\n Prime + Prime<\/td>\n ?<\/td>\n <\/td>\n No conversion<\/td>\n<\/tr>\n \n ′<\/td>\n 2032<\/td>\n Prime<\/td>\n ′<\/td>\n 2032<\/td>\n Prime<\/td>\n ‘<\/td>\n 0027<\/td>\n Apostrophe<\/td>\n<\/tr>\n \n ‵<\/td>\n 2035<\/td>\n Reverse prime<\/td>\n ‵<\/td>\n 2035<\/td>\n Reverse prime<\/td>\n `<\/td>\n 0060<\/td>\n Grave accent<\/td>\n<\/tr>\n \n ‹<\/td>\n 2039<\/td>\n Single left-pointing angle quotation mark<\/td>\n ‹<\/td>\n 2039<\/td>\n Single left-pointing angle quotation mark<\/td>\n <<\/td>\n 003C<\/td>\n Less-than sign<\/td>\n<\/tr>\n \n «<\/td>\n 00AB<\/td>\n Left-pointing double angle quotation mark<\/td>\n «<\/td>\n 00AB<\/td>\n Left-pointing double angle quotation mark<\/td>\n <<\/td>\n 003C<\/td>\n Less-than sign<\/td>\n<\/tr>\n \n —<\/td>\n 2014<\/td>\n Em-dash<\/td>\n —<\/td>\n 2014<\/td>\n Em-dash<\/td>\n –<\/td>\n 002D<\/td>\n Hyphen-minus<\/td>\n<\/tr>\n ‼<\/td>\n 203C<\/td>\n Double exclamation mark<\/td>\n !!<\/td>\n 0021 0021<\/td>\n Exclamation mark + Exclamation mark<\/td>\n ?<\/td>\n <\/td>\n No conversion<\/td>\n<\/tr>\n<\/table>\n