{"id":463,"date":"2014-07-18T07:00:00","date_gmt":"2014-07-18T07:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/2014\/07\/18\/the-time-one-of-my-colleagues-debugged-a-line-of-business-application-for-a-package-delivery-service\/"},"modified":"2014-07-18T07:00:00","modified_gmt":"2014-07-18T07:00:00","slug":"the-time-one-of-my-colleagues-debugged-a-line-of-business-application-for-a-package-delivery-service","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20140718-00\/?p=463","title":{"rendered":"The time one of my colleagues debugged a line-of-business application for a package delivery service"},"content":{"rendered":"<p>\nBack in the days of Windows&nbsp;95 development,\none of my colleagues debugged\na line-of-business application for a major delivery service.\nThis was a program that the company gave to its top-tier\nhigh-volume customers,\nso that they could place and track their orders directly.\nAnd by <i>directly<\/i>,\nI mean that the program\ndialed the modem\n(since that was how computers communicated with each other back then)\nto contact\nthe delivery service&#8217;s mainframe\n(it was all mainframes back then)\nand upload the new orders\nand download the status of existing orders.&sup1;\n<\/p>\n<p>\nVersion 1.0 of the application had a notorious bug:\nNinety days after you installed the program,\nit stopped working.\nThey forgot to remove the beta expiration code.\nI guess that&#8217;s why they have a version 1.01.\n<\/p>\n<p>\nAnyway, the bug that my colleague investigated\nwas that if you entered a particular type of\norder with a particular set of options\nin a particular way,\nthen the application crashed your system.\nSetting up a copy of the application\nin order to replicate the problem\nwas itself a bit of an ordeal,\nbut that&#8217;s a whole different story.\n<\/p>\n<p>\nOkay, the program is set up, and yup, it crashes exactly\nas described when run on Windows&nbsp;95.\nActually, it also crashes exactly as described when\nrun on Windows&nbsp;3.1.\nThis is just plain an application bug.\n<\/p>\n<p>\nHere&#8217;s why it crashed:\nAfter the program dials up the mainframe to submit the order,\nit tries to refresh the list of orders that have yet to be delivered.\nThe code that does this assumes that the list of undelivered orders\nis the control with focus.\nBut if you ask for labels to be printed,\nthen the printing code changes focus in order to\ndisplay the &#8220;Please place the label on the package\nexactly like this&#8221; dialog,\nand as a result, the refresh code can&#8217;t find the undelivered order list\nand crashes on a null pointer.\n(I&#8217;m totally making this up, by the way.\nThe details of the scenario aren&#8217;t important to the story.)\n<\/p>\n<p>\nOkay, well, that&#8217;s no big deal.\nA null pointer fault should just put up the\nUnrecoverable Application Error dialog box\nand close the program.\nWhy does this particular null pointer fault crash the entire system?\n<\/p>\n<p>\nThe developers of the program saw that their refresh code\nsometimes crashed on a null pointer,\nand instead of fixing it by actually fixing the code so it\ncould find the list of undelivered orders even if it didn&#8217;t\nhave focus,\nor fixing it by adding a null pointer check,\nthey fixed it by adding a null pointer exception handler.\n(I wish to commend myself for\nresisting the urge to put the word <i>fixed<\/i> in quotation marks\nin that last sentence.)\n<\/p>\n<p>\nNow, 16-bit Windows didn&#8217;t have structured exception handling.\nThe only type of exception handler was a global exception handler,\nand this wasn&#8217;t just global to the process.\nThis was global to the entire system.\nYour exception handler was called for every exception everywhere.\nIf you screwed it up, you screwed up the entire system.\n(I think you can see where this is going.)\n<\/p>\n<p>\nThe developers of the program converted their global exception handler\nto a local one by going\nto every function that had a\n&#8220;We seem to crash on a null pointer and I\ndon&#8217;t know why&#8221; bug and making these changes:\n<\/p>\n<pre>\nextern jmp_buf caught;\nextern BOOL trapExceptions;\nvoid scaryFunction(...)\n{\n <font COLOR=\"blue\">if (setjmp(&amp;caught)) return;\n trapExceptions = TRUE;<\/font>\n ... body of function ...\n <font COLOR=\"blue\">trapExceptions = FALSE;<\/font>\n}\n<\/pre>\n<p>\nTheir global exception handler checks the <code>trapExceptions<\/code>\nglobal variable,\nand if it is <code>TRUE<\/code>,\nthey set it back to <code>FALSE<\/code> and\ndo a <code>longjmp<\/code> which sends control\nback to the start of the function,\nwhich detects that something bad must have happened and just\nreturns out of the function.\n<\/p>\n<p>\nYes, things are kind of messed up as a result of this.\nYes, there is a memory leak.\nBut at least their application didn&#8217;t crash.\n<\/p>\n<p>\nOn the other hand, if the global variable is <code>FALSE<\/code>,\nbecause their application crashed in some other function that\ndidn&#8217;t have this special protection,\nor because some other totally unrelated application crashed,\nthe global exception handler decided to exit the application\nby running around freeing all the DLLs and memory associated with their\napplication.<\/p>\n<p><p>\nOkay, so far so good, for certain values of <i>good<\/i>.\n<\/p>\n<p>\nThese system-wide exception handlers had to be written in assembly\ncode because they were dispatched with a very strange calling\nconvention.\nBut the developers of this application\ndidn&#8217;t write their system-wide exception handler\nin assembly language.\nTheir application was written in MFC,\nso they just went to Visual C++ (as it was then known),\nclicked through some <i>Add a Windows hook<\/i> wizard,\nand got some generic <code>HOOKPROC<\/code>.\n(I don&#8217;t know if Visual C++ actually had an\n<i>Add a Windows hook<\/i> wizard;\nthey could just have copied the code from somewhere.)\nNevermind that these system-wide exception handlers are not\n<code>HOOKPROC<\/code>s, so the function has the wrong prototype.\nWhat&#8217;s more, the code they used marked the hook function\nas <code>__loadds<\/code>.\nThis means that the function saves the previous value of the\n<code>DS<\/code> register on entry,\nthen changes the register to point to the application&#8217;s data,\nand on exit, the function restores the previous value of <code>DS<\/code>.\n<\/p>\n<p>\nOkay, now we&#8217;re about to enter the\n<a HREF=\"http:\/\/gideonsway.wordpress.com\/2011\/04\/06\/what-is-a-movie-set-piece\/\">\nset piece<\/a> at the end of the movie:\nOur hero&#8217;s fear of spiders,\nhis girlfriend&#8217;s bad ankle from an old soccer injury,\nthe executive toy on the villain&#8217;s desk,\nand all the other tiny little clues\ndropped in the previous ninety minutes\ncome together to form an enormous chain reaction.\n<\/p>\n<p>\nThe application crashes on a null pointer.\nThe system-wide custom exception handler is called.\nThe crash is not one that is being protected by the global variable,\nso the custom exception handler frees the application from memory.\nThe system-wide custom exception handler now returns,\nbut wait, what is it returning to?\n<\/p>\n<p>\nThe crash was in the application,\nwhich means that the <code>DS<\/code> register\nit saved on entry to the custom exception handler\npoints to the application&#8217;s data.\nThe custom exception handler freed the application&#8217;s data\nand then returned, declaring the exception handled.\nAs the function exited,\nit tried to restore the original <code>DS<\/code> register,\nbut the CPU said,\n&#8220;Nice try,\nbut that is not a valid value for the <code>DS<\/code> register\n(because you freed it).&#8221;\nThe CPU reported this error by (dramatic pause) raising an exception.\n<\/p>\n<p>\nThat&#8217;s right, the system-wide custom exception handler\ncrashed with an exception.\n<\/p>\n<p>\nOkay, things start snowballing.\nThis is the part of the movie where the director uses\nquick cuts between different locations,\n<a HREF=\"http:\/\/www.thelonelyisland.com\/video\/cool-guys-explosions\">\nmaybe with a little slow motion thrown in<\/a>.\n<\/p>\n<p>\nSince an exception was raised,\nthe custom exception handler is called recursively.\nEach time through the recursion, the custom exception handler\nfrees all the DLLs and memory associated with the application.\nBut that&#8217;s okay, right?\nBecause the second and subsequent times,\nthe memory was already freed,\nso the attempts to free them again will just fail\nwith an invalid parameter error.\n<\/p>\n<p>\nBut wait, their list of DLLs associated\nwith the application included\n<code>USER<\/code>,\n<code>GDI<\/code>,\nand\n<code>KERNEL<\/code>.\nNow, Windows is perfectly capable of unloading dependent DLLs\nwhen you unload the main DLL,\nso when they unloaded their main program, the kernel already\ndecremented\nthe usage count on\n<code>USER<\/code>,\n<code>GDI<\/code>,\nand\n<code>KERNEL<\/code> automatically.\nBut they apparently didn&#8217;t trust Windows to do this,\nbecause after all,\nit was Windows that was causing their application to crash,\nso they took it upon themselves to free those DLLs manually.\n<\/p>\n<p>\nTherefore, each time through the loop,\nthe usage counts for\n<code>USER<\/code>,\n<code>GDI<\/code>,\nand\n<code>KERNEL<\/code> drop by one.\n<i>Zoom in on the countdown clock on the ticking time bomb.<\/i>\n<\/p>\n<p>\n<i>Beep beep beep beep beep<\/i>.\nThe reference count finally drops to zero.\nThe window manager, the graphics subsystem,\nand the kernel itself have all been unloaded from memory.\nThere&#8217;s nothing left to run the show!\n<\/p>\n<p>\nBoom, bluescreen.\nHot flaming death.\n<\/p>\n<p>\nThe punch line to all this is that whenever you call the company&#8217;s\nproduct\nsupport line and describe a problem you encountered,\ntheir response is always,\n&#8220;Yeah, we&#8217;re really sorry about that one.&#8221;\n<\/p>\n<p>\n<b>Bonus chatter<\/b>:\nWhat is that <i>whole different story<\/i>\nmentioned near the top?\n<\/p>\n<p>\nWell, when the delivery service sent the latest version\nof the software to the Windows&nbsp;95 team,\nthey also provided an account number to use.\nMy colleague used that account number to\ntry to reproduce the problem,\nand since the problem occurred only after the order was\nsubmitted,\nshe would have to submit delivery requests,\nsay for a letter to be picked up\nfrom 221B Baker Street\nand delivered to 62 West Wallaby Street,\nor maybe for a 100-pound package of radioactive material\nto be picked up from\n1600 Pennsylvania Avenue\nand delivered to\n10 Downing Street.\n<\/p>\n<p>\nAfter about two weeks of this,\nmy colleague got a phone call from Microsoft&#8217;s\nshipping department.\n&#8220;What the heck are you doing?&#8221;\n<\/p>\n<p>\nIt turns out that the account number my colleague was given\nwas Microsoft&#8217;s own corporate account number.\nAs in a <i>real live account<\/i>.\nShe was inadvertently prank-calling the delivery company\nand sending actual trucks all over the\ncountry to pick up nonexistent letters and packages.\nMicrosoft&#8217;s shipping department and people\nfrom the delivery service&#8217;s headquarters\nwere frantic trying to trace where all the bogus\norders were coming from.\n<\/p>\n<p>\n&sup1;\nMind you, this sort of thing is the stuff that average Joe\ncustomers can do while still in their pajamas,\nbut back in those days,\nit was a feature that only top-tier customers had access to,\nbecause, y&#8217;know, mainframe.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Back in the days of Windows&nbsp;95 development, one of my colleagues debugged a line-of-business application for a major delivery service. This was a program that the company gave to its top-tier high-volume customers, so that they could place and track their orders directly. And by directly, I mean that the program dialed the modem (since [&hellip;]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[2],"class_list":["post-463","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-history"],"acf":[],"blog_post_summary":"<p>Back in the days of Windows&nbsp;95 development, one of my colleagues debugged a line-of-business application for a major delivery service. This was a program that the company gave to its top-tier high-volume customers, so that they could place and track their orders directly. And by directly, I mean that the program dialed the modem (since [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/463","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=463"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/463\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=463"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=463"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=463"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}