{"id":23713,"date":"2008-01-24T10:00:00","date_gmt":"2008-01-24T10:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/2008\/01\/24\/its-one-thing-to-say-somebody-should-do-this-but-doing-it-is-another-matter\/"},"modified":"2008-01-24T10:00:00","modified_gmt":"2008-01-24T10:00:00","slug":"its-one-thing-to-say-somebody-should-do-this-but-doing-it-is-another-matter","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20080124-00\/?p=23713","title":{"rendered":"It's one thing to say "somebody should do this", but doing it is another matter"},"content":{"rendered":"

\nA common response when I describe a programming error is,\n““Programs should have to pass a test that includes testing for\nthis case<\/a>.”\nThe case could be a program mishandling a message,\na program responding incorrectly to IUnknown::QueryInterface<\/code>,\nwhatever.\nBut these suggestions fall into the same trap that I see\n\nwhen grading student essays<\/a>:\nThey’re good with the what<\/i> but very weak on the how<\/i>.\n<\/p>\n

\nSaying “Somebody should do X” is easy,\nbut without some sort of suggestion as to how that could be accomplished,\nthe suggestion rarely gets off the drawing board.\nIt’s like saying\n“Somebody should solve world hunger.”\n<\/p>\n

\nLet’s look at that first example again.\nThe topic at hand was window procedures which fail to pass unhandled messages\nto the DefWindowProc<\/code> function.\nHow would one “test for this”?\nWould the test walk through every code path in the program that\ncreates a window, and then send each of those windows a fake\nWM_QUERYENDSESSION<\/code> or a fake\nWM_APPCOMMAND<\/code> message to see what happens?\nFirst of all, it’s unclear how a test could exercise all the window-creation\ncode paths of a program without insider knowledge of that program.\nTherefore, this test would have to be written by the authors of\nthe program.\n<\/p>\n

\nNext, even if you sent the message and saw that the message\nwas passed to the DefWindowProc<\/code> function,\nthat wouldn’t actually prove that the message was handled\nproperly.\nMaybe the window procedure for a window goes something like this:\n<\/p>\n

\n...\ncase WM_QUERYENDSESSION:\n if (GetTickCount() \/ 1000 % 2) return 0;\n return DefWindowProc(hwnd, uMsg, wParam, lParam);\n<\/pre>\n
\nEven if you managed to get this window created,\nif you send it a fake WM_QUERYENDSESSION<\/code>,\nyou’ll catch the issue only half the time.\nIt’s not enough just to exercise every window procedure;\nyou also have to exercise every code path.\n<\/p>\n
\nBut wait, there’s more.\nWhat if the program really wanted to prevent the user from logging off?\n<\/p>\n
\n...\ncase WM_QUERYENDSESSION:\n if (FileHasBeenEditedSinceLastSave()) {\n  switch (MessageBox(hwnd,\n             TEXT(\"Save changes before exiting?\"),\n             TEXT(\"Title\"), MB_YESNOCANCEL)) {\n  case IDYES: Save(); break;\n  case IDCANCEL: return 0; \/\/ user cancelled logoff\n  }\n }\n return DefWindowProc(hwnd, uMsg, wParam, lParam);\n<\/pre>\n
\nIn this case, there is a code path that cancels the logoff,\nand it is legitimate, since it was done as the result of\na user decision.\nYour test would somehow have to know this and consider\nthat case to be a pass and not a failure.\nThis sort of reasoning is hardly something that a generic\ntest suite can do; it has to be tailored for each program.\n<\/p>\n
\nIt’s one thing to say that something should be tested,\nbut without an idea as to how<\/i> it should be tested,\nthe suggestion is much less valuable.\nI may as well say,\n“Programs should have to pass a test that verifies that there\nare no bugs.”<\/p>\n","protected":false},"excerpt":{"rendered":"
A common response when I describe a programming error is, ““Programs should have to pass a test that includes testing for this case.” The case could be a program mishandling a message, a program responding incorrectly to IUnknown::QueryInterface, whatever. But these suggestions fall into the same trap that I see when grading student essays: They’re […]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[26],"class_list":["post-23713","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-other"],"acf":[],"blog_post_summary":"
A common response when I describe a programming error is, ““Programs should have to pass a test that includes testing for this case.” The case could be a program mishandling a message, a program responding incorrectly to IUnknown::QueryInterface, whatever. But these suggestions fall into the same trap that I see when grading student essays: They’re […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/23713","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=23713"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/23713\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=23713"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=23713"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=23713"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}