Adventures in application compatibility: Querying for an internal interface
Raymond Chen
An application compatibility report came in for a program that crashed during installation. The primary installer created a temporary directory and then ran a secondary installer from that temporary directory. So far, that’s standard behavior to avoid operating from a disease-ridden hot tub. What made it more frustrating to debug is that the secondary installer in turn runs a tertiary installer, and it was the tertiary installer that was crashing. (At least it stopped at the tertiary installer. Otherwise it’s installers all the way down.)
Some debugging of the tertiary installer revealed that it was trying to pin itself to the taskbar automatically on install. This is one of those things that installers love to do, probably second only to silently taking over popular file extensions. This particularly program’s approach was to find the vtable for the internal interface that Explorer uses to pin items to the taskbar, and then call the fifth function in the vtable.
The application compatibility issue was that we changed the signature of the fifth function in the vtable, so they were calling the function with the old signature, and the implementation was therefore misinterpreting the parameters, and as a bonus insult, it also corrupted the stack because the number of parameters also changed.
The solution was to restore the function to its original signature and have it just return without doing anything, decorated with comments to ensure that the method stays at the same slot in the vtable for eternity. Meanwhile, the new functionality was moved to a different location. (I’m not telling where.)
Light
Dark
18 comments
I realize it’s not the topic of this article, but since you link to the article about file extensions…. It would be nice if Microsoft followed it’s own advice.
I’m looking at you MS Edge.
I have to use Edge as web-based interface to an internal data management tool. Even though in Default Apps settings I have PDF set to a third party tool Edge will decide that it is the PDF viewer after doing a save-as on a downloaded file. As bad as it sounds, IE11 was the best when it came to handling downloaded PDFs.
Go to the Edge settings. Then search for `PDF documents`. If `Always download PDF files` is checked then uncheck it. By default if Edge downloads a PDF then it auto-opens it in Edge. See if this solves your problem.
Gah, I remember that, a few years ago when Edge was first introduced it really, really, really wanted to be the one and only PDF viewer and kept taking over all PDF management even though it did a really lousy job at it. This got so bad that Adobe did a walkthrough in their software when you ran it showing you all the steps you needed to take to disable this hijacking. I kept wondering why PDF viewing and printing was suddenly so broken and realised that Windows had seized control of it again…
Related reading: Why do my PDF file associations get reset every time I restart?
Raymond is on record as having commented that many of the sins he blames on “Contoso” are actually committed by Microsoft. No, he isn’t say which is which… So no company is safe from his disapproval, not even his own! He’s like Darkwing Duck…
The shell team have also decided that certain menu items only appear if your process is Explorer.exe, a bit unfair for 3rd-party file explorers using IExplorerBrowser legitimately.
Specially since 3rd-party file explorers are the only usable way to write extensions in .NET and WinRT, as the shell keeps stuck in good old COM.
Was it not possible to shim/patch the installer? Or is that much harder than changing explorer?
Rule of multiple offenders. Especially with hacky code like this that just begs to be shared around.
I thought pinning stuff to the Win7+ taskbar (“Superbar”) was never meant to be controlled programmatically? Like that’s up to the user to decide what is pinned, as with pinned apps in the old Win7+ start menu? Are some applications so important that Microsoft ignores their purposefully set limitations and fixes compatibility if these apps work around them – or do I just badly remember the set limitations…?
Notice that the “fix” changes the function to a nop. The limitation is still in force.
Oh, I missed that detail in my rage over that app. That kinda makes more sense. Yet I’d personally be unsatisfied without such programs crashing from doing this bogus and code getting yucky because of them. But yeah, we know who’d get the blame for the crash eventually…
I know of at least one program that at least only offers to programmatically pin itself to the taskbar for you, so still leaving it up to the user (I declined the pin). Its approach to pinning itself is at least somewhat less hacky than the installer in the article.
This is my favourite kind of oldnewthing story. What i always wonder though is why Microsoft don’t open up these “forbidden” APIs, but embeds user confirmation dialogs in them? Sure, it’s bad sport to mix UI and functions but still.
It was probably moved to 6th vtable entry.
Because developers abuse them, no matter how user hostile that is. Want the user to say Yes to pinning your app?
While (app_is_not_pinned) { call request_app_to_be_pinned(); }And then just spam the user with acceptance dialogue until they just click Yes. There are already enough out there that guide users through a process to change global settings and keep pestering if you don’t.
You wouldn’t even need to pester the user like that – unless the dialog is isolated like the UAC dialog, the application can just click the Yes button itself via a window message or UI Automation. Or the application could just patch the internal function so that it doesn’t open a dialog in the first place, or hook CreateDialog or whatever API call gets used to create the dialog.
Yeah, but for such hostile apps it’s better if they patch out known APIs than to patch some random vtables by scanning memory
I bet someone got a really nice bonus for that “feature”.