During the development of Windows Vista, the application comaptibility team traced a bunch of issues back to people corrupting the stack by using the rundll32 program to call functions that were not designed to be called by rundll32.

The problems were often subtle. For example, a batch file which used rundll32 incorrectly ended up hanging because the rundll32 process never returned. The misaligned stack resulted in registers being restored from the stack incorrectly, and then the cleanup code inside rundll32 ends up getting confused and wedging itself. The programs got away with it on previous versions of Windows by sheer luck. The version of the compiler used by Windows Vista contains different optimizations, and it ended up arranging stack variables and using registers differently, and what in previous versions of Windows was some corruption that went largely unnoticed became corruption that resulted in the program getting stuck in an infinite loop. Lucky no longer.

I was asked to come up with a solution for this problem, to fix the rundll32 program so it was more resilient to people who used it incorrectly. To fix other people’s bugs for them.

The solution: Before calling the function, push a hundred bytes of garbage onto the stack (in case the called function pops too many bytes off the stack) and save the stack pointer in a global variable. After the function returns, restore the stack pointer, in case the called function pops too many or too few bytes off the stack. I think I may even have saved the processor registers in global variables, I forget.

Do not consider this free license to continue abusing the rundll32 program. When the pet store opens on Sundays, that doesn’t mean that it’s okay to keep throwing garbage on the sidewalk.