{"id":112002,"date":"2026-01-21T07:00:00","date_gmt":"2026-01-21T15:00:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/oldnewthing\/?p=112002"},"modified":"2026-01-21T08:47:42","modified_gmt":"2026-01-21T16:47:42","slug":"20260121-00","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20260121-00\/?p=112002","title":{"rendered":"On the proper usage of a custom Win32 dialog class"},"content":{"rendered":"

Some time ago, I discussed custom dialog classes<\/a>. You can specify that a dialog template use your custom dialog class by putting the custom class’s name in the CLASS<\/tt> statement of the dialog template. A customer tried doing that but it crashes with a stack overflow.<\/p>\n

\/\/ Dialog template\r\n\r\nIDD_AWESOME DIALOGEX 0, 0, 170, 62\r\nSTYLE DS_SHELLFONT | DS_MODALFRAME | WS_POPUP | WS_CAPTION\r\nCAPTION \"I'm so awesome\"\r\nCLASS \"MyAwesomeDialog\"\r\nFONT 8, \"MS Shell Dlg\", 0, 0, 0x1\r\nBEGIN\r\n    ICON            IDI_AWESOME,IDC_STATIC,14,14,20,20\r\n    LTEXT           \"Whee!\",IDC_STATIC,42,14,114,8\r\n    DEFPUSHBUTTON   \"OK\",IDOK,113,41,50,14,WS_GROUP\r\nEND\r\n\r\n\/\/ Custom dialog class procedure\r\n\/\/ Note: This looks ugly but that's not the point.\r\nLRESULT CALLBACK CustomDlgProc(HWND hwnd, UINT message,\r\n                               WPARAM wParam, LPARAM lParam)\r\n{\r\n    if (message == WM_CTLCOLORDLG) {\r\n        return (LRESULT)GetSysColorBrush(COLOR_INFOBK);\r\n    }\r\n    return DefDlgProc(hwnd, message, wParam, lParam);\r\n}\r\n\r\nvoid Test()\r\n{\r\n    \/\/ Register the custom dialog class\r\n    WNDCLASS wc{};\r\n    GetClassInfo(nullptr, WC_DIALOG, &wc);\r\n    wc.lpfnWndProc = CustomDlgProc;\r\n    wc.lpszClassName = TEXT(\"MyAwesomeDialog\");\r\n    RegisterClass(&wc);\r\n\r\n    \/\/ Use that custom dialog class for a dialog\r\n    DialogBox(hInstance, MAKEINTESOURCE(IDD_AWESOME), hwndParent,\r\n              CustomDlgProc);\r\n}\r\n<\/pre>\n
Do you see the problem?<\/p>\n
The problem is that the code uses the CustomDlgProc<\/code> function both as a window procedure and<\/i> as a dialog procedure.<\/p>\n
When a message arrives, it goes to the window procedure. This rule applies regardless of whether you have a traditional window or a dialog. If you have a standard dialog, then the window procedure is Def\u00adDlg\u00adProc<\/code>, and that function calls the dialog procedure to let it respond to the message. If the dialog procedure declines to handle the message, then the Def\u00adDlg\u00adProc<\/code> function does some default dialog stuff.<\/p>\n
Creating a custom dialog class means that you want a different window procedure for the dialog, as if you had subclassed the dialog. The custom window procedure typically does some special work, and then it passes messages to Def\u00adDlg\u00adProc<\/code> when it wants normal dialog behavior.<\/p>\n
If you use the same function as both the window procedure and the dialog procedure, then when the function (acting as a window procedure) passes a message to Def\u00adDlg\u00adProc<\/code>, the Def\u00adDlg\u00adProc<\/code> function will call the dialog procedure, which is also<\/i> Custom\u00adDlg\u00adProc<\/code>. That function doesn’t realize that it’s now being used as a dialog procedure, so it is expected to return TRUE<\/code> or FALSE<\/code> (depending on whether it decided to handle the message). It thinks it is still a window procedure, so it passes the message to Def\u00adDlg\u00adProc<\/code>, and the loop continues until you overflow the stack.<\/p>\n
The idea behind custom dialog classes is that you have some general behavior you want to apply to all the dialogs that use that class. For example, maybe you want them all to use different default colors, or you want them all to respond to DPI changes the same way. Instead of replicating the code in each dialog procedure, you can put it in the dialog class window procedure.<\/p>\n
But even if you are using a custom dialog class, your dialog procedure should still be a normal dialog procedure. That dialog procedure is the code-behind for the dialog template, initializing the controls in the template, responding to clicks on the controls in the template, and so on.<\/p>\n","protected":false},"excerpt":{"rendered":"
You are replacing the window procedure, not the dialog procedure.<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-112002","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"
You are replacing the window procedure, not the dialog procedure.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/112002","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=112002"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/112002\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=112002"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=112002"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=112002"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}