A customer reported a memory leak in Windows that occurred when they called RegisterDragDrop followed by RevokeDragDrop. They included a time travel trace of a sample program that demonstrated the problem. (Though for some reason, they didn’t include the program itself; just the time travel trace.)
Now, it is strange that there would be a memory leak if you call RegisterDragDrop followed by RevokeDragDrop, seeing as this pattern is used heavily by thousands of applications, including many parts of Windows itself, so if there were a memory leak inherent in the pattern, you’d think it’d have been reported by now.
I suspected that there was something special about their sample program.
Some time ago, I noted that No, really, you need to pass all unhandled messages to DefWindowProc. And that was the source of the problem.
Debugging through the time travel trace showed that yes, they did call RegisterDragDrop, and then they did call RevokeDragDrop. But there’s more going on. When the window receives a WM_DESTROY message, it cleans up all its state. And for any messages that arrive after WM_DESTROY, the window procedure goes looking for its special state and doesn’t see it, so it gives up and just returns 0 without passing the message to DefWindowProc.
Oops.
If the window procedure can’t figure out what to do, it should pass all messages to DefWindowProc. In this case, it’s important because some of those messages are cleanup messages, and one of the things those cleanup messages do is free the last few fragments of memory still hanging around.
Bonus chatter: But if I register a drop target, and then revoke it, shouldn’t the revoke free all the memory that was allocated by the register call?
There’s no requirement that registering something and then unregistering it will immediately free all the memory associated with the registration. The system is allowed to cache stuff that it thinks will be needed again.
In this case, what happened is that the RegisterDragDrop function uses an infrastructure that is shared by many components. That infrastructure is created and attached to the window the first time anybody needs it, and it is cleaned up when the window is destroyed. The memory isn’t leaked. It’s just cached on the window, waiting to be used by another operation. And the cache is destroyed when the window is destroyed.
But it assumes that you give DefWindowProc a chance to do that cleanup.
msdn seems to imply that DefWindowProc(WM_DESTROY) is optional: If an application processes this message, it should return zero.
mandatory application independent cleanups mentioned in this post should be put into whatever unconditionally runs after WindowProc(WM_DESTROY) returns, e.g. DestoryWindow, instead of DefWindowProc
That is because WM_DESTROY doesn't free memory. It is sent to a window before it is destroyed for window specific cleanup. I wouldn't be surprised if the WM_DESTROY handler in DefWindowProc is just to return 0. Also, just because a message states that "it should return zero" then it doesn't mean that it is optional. The way to think of it is that those messages have no return, so 0 is used as a safe value.
On the other hand, WM_NCDESTROY is documented to free memory. So this is a message that must be passed to DefWindowProc, even if you use...
There’s a thing called
“DwmDefWindowProc”
What is this for and should this be considered an alternative to DefWindowProc?
DwmDefWindowProc is there for if you are drawing a custom window frame, it is documented to do the hit testing that the system does. So no, it is not an alternative, it supplements DefWindowProc in a specific situation.