On the importance of making sure WaitForInputIdle doesn’t think you’re idle, episode 1

A customer had a program that was originally designed back when DDE ruled the land, and it continues to support DDE for (yup) backward compatibility.

Today and tomorrow are going to deal with DDE issues, but not because DDE is actually anything recommended. Quite the contrary: Please feel free to stop using DDE. DDE is so obsolete that when somebody has a DDE question, it usually lands in the lap of some old-timer who starts the answer with “Kids these days…” and ends it with “I’m going to write a blog entry about this.”

Anyway, the customer was making some changes to the program’s splash screen, and one of the changes broke DDE. “With my change, launching a file from Explorer displays the dreaded There was a problem sending the command to the program. What should I be looking out for?”

Recall that launching documents via DDE is done by first looking for a DDE server and if none is found, launching a server manually and trying again. And the retry occurs after the manually-launched server goes input idle; in other words, after the shell launches the server, the shell calls Wait­For­Input­Idle, and then when that call returns, the shell goes looking for the DDE server. If it can’t find the server the second time, you get the There was a problem sending the command to the program error.

We saw earlier that the Wait­For­Input­Idle function declares the process idle once any thread is “waiting for user input with no input pending.” In the customer’s case, the issue was not a background thread that went idle, but rather that their main UI thread was performing a Co­Create­Instance call to a local server. Cross-process method calls are performed by marshaling the request to the server, then waiting for the server to return a response. And in a single-threaded apartment, COM pumps messages while waiting.

It is that COM message pump that was causing the process to be marked idle.

The customer needs to get the DDE server window created and ready to accept incoming commands before the program does any cross-thread COM operations from a single-threaded apartment.

Okay, that was basically a warm-up. Next time, a different angle on failed DDE startup.