June 26th, 2026
likemind blown6 reactions

The case of the DLL that was not present in memory despite not being formally unloaded, part 2

Last time, we looked at crashes caused by a DLL being removed from memory behind everybody’s back, causing crashes when somebody tried to call into that no-longer-there DLL that everybody thought was still there.

A colleague of mine who was looking at other crashes coming from this process found that most of those other crashes were also of the form “a data structure was corrupted because somebody wrote the single byte 01 into it.” That piece of information made everything fall into place for my side of the investigation.

We saw earlier that the bottom bit of the HMODULE is set for datafile module handles. Therefore, if one of these stray 01 bytes happens to overwrite the bottom byte of an existing HMODULE handle, that turns it into a (fake) datafile module handle. And then, during process destruction, a component dutifully cleans up the DLLs they loaded by freeing them (say because they were stored in an RAII type like wil::unique_hmodule), the code will pass this (fake) datafile module handle to Free­Library. The Free­Library function sees the bottom bit set and says, “Oh, this must be the handle to a module that was loaded via LOAD_LIBRARY_AS_DATAFILE,” so it frees it as a datafile.

Freeing a datafile module means undoing the steps that were taken when the module was loaded as a datafile: Unmapping the DLL from memory. In particular, loading a module as a datafile does not add the DLL to the list of DLLs that were loaded as code; therefore, unloading a datafile module doesn’t remove it from that list. As far as the DLL list is concerned, the DLL is still in memory.

A one-bit error caused the code to lie and attempt to free a module handle that did not correspond to a Load­Library call, resulting in mass havoc.

The “DLL unmapped from memory” crash is just an alternate manifestation of the “somebody is writing 01 bytes to places they shouldn’t” bug. The original bug had a larger bucket spray than we initially thought.

The good news is that all of the crashes have funneled down to a single bug. The bad news is that you now have to debug this one memory corruption bug.

Unfortunately, at the time of this writing, the root memory corruption bug in the third party program has yet to be identified. We don’t know whether it’s coming from an operating system component or from the program itself. Though the fact that it appears to occur only in one process, where it sprays across multiple modules, suggests that it’s a problem with that program, or that there’s something peculiar about how this specific process uses the system.

If you look at the original stack trace, you can see that the problem is occurring at process termination. That’s probably why the problem has lurked for so long: Crashes at exit often go unnoticed because there is no end-user loss of functionality. The user was finished with the program anyway. Whether it exits cleanly or with a crash doesn’t affect the user much.

Sorry. Not all stories have a happy ending.

Topics

Author

Raymond has been involved in the evolution of Windows for more than 30 years. In 2003, he began a Web site known as The Old New Thing which has grown in popularity far beyond his wildest imagination, a development which still gives him the heebie-jeebies. The Web site spawned a book, coincidentally also titled The Old New Thing (Addison Wesley 2007). He occasionally appears on the Windows Dev Docs Twitter account to tell stories which convey no useful information.

21 comments

Discussion is closed. Login to edit/delete existing comments.

Sort by :
  • Steve

    There are several operating system components that exhibit similar bugs, many of which appear to have been introduced relatively recently. One example is the FreeConsole API on Windows 10 and Windows 11 - FreeConsole initially destroys the console/file/pipe etc... handles cached in the undocumented PEB, but it fails to clear the corresponding PEB fields afterward.

    As a result, subsequent console APIs continue to treat these stale handle values as valid. Because Windows aggressively recycles handle identifiers, a completely unrelated object is assigned the same handle value that was previously used by the console. The console subsystem suddenly, and unknowingly resumes using...

    Read more
  • Yexuan Xiao

    I think the problem lies in UnbindCrtHandles. Console handles are not managed by the CRT, and the CRT does not change their state. Therefore, if UnbindCrtHandles destroys the handles, FreeConsole will fail. You can reach out to the Windows Console team at github.com/microsoft/terminal, they don’t own the UCRT, but they do own the console host, the driver, and FreeConsole.

    • Todd Bandrowsky · Edited

      Thank you, your insights have been awesome and I appreciate you!

      UnbindCrtHandles comes from MS example. So at a high level, I have this console handle that I get from somewhere, attach it to the CRT, and the CRT, when I go to unattach it, deletes the handles. Is freopen_s not the thing to do here?

      Or maybe is it this approach of attaching and unattaching handles is simply wrong.

      Like I thought C file handles were like a structure somewhere of a fixed size, and you could just plop in your OS handle into that, or, set it to null,...

      Read more
      • Yexuan Xiao

        I have read some materials and guess that you used freopen("CONOUT$", "w", stdout);. Perhaps the issue lies in the management of "CONOUT$", or in how the CRT handles stdin. In fact, Microsoft has open-sourced part of the UCRT, which fully includes the implementations related to fd and FILE*. You can find them at github.com/huangqinjin/ucrt and investigate how the UCRT manages these to determine whether the UCRT is the root cause.

        Regardless of the reason, I believe the UCRT really messed up on this matter. It should have provided proper initialization and uninitialization functions for stdin/stdout/stderr directly, rather than leaving users to...

        Read more
  • Eric Brown

    No. If you have dependencies on DLLs that have complex dependencies of their own, the only reliable way to shut down is to manage the cleanup yourself.

    Relying on the C runtime does not guarantee that things get cleaned up in the right order.

  • Todd Bandrowsky · Edited

    I think is the console, or perhaps, the stdin,stdout file handles on the console. Probably all the recent changes that made the console not suck on Windows, which I love and adore and bless Microsoft for, may have broken this.

    I have a few steps that can reliably reproduce this kind of problem.

    I have an application that is a Windows Desktop application, using WinMain, in C++.
    My main window gets a DirectX surface, which, may or may not be relevant.
    Then I have a Console, which I create like so.

    <code>

    Now I do all my Windows and DirectX stuff and it...

    Read more