Last time, we looked at adding or removing a handle from an active Msg­Wait­For­Multiple­Objects, and observed that we could send a message to both break out of the wait and update the list of handles. But what if the other thread is waiting in a Wait­For­Multiple­Objects? You can’t send a message since Wait­For­Multiple­Objects doesn’t wake for messages.

You can fake it by using an event which means “I want to change the list of handles.” The background thread can add that handle to its list, and if the “I want to change the list of handles” event is signaled, it updates its list.

One of the easier ways to represent the desired change is to maintain two lists, the “active” list (the one being waited on) and the “desired” list (the one you want to change it to). The background thread can make whatever changes to the “desired” list it wants, and then it signals the “changed” event. The waiting thread sees that the “changed” event is set and copies the “desired” list to the “active” list. This copying needs to be done with Duplicate­Handle because the background thread might close a handle in the “desired” list, and we can’t close a handle while it is being waited on.

wil::unique_handle duplicate_handle(HANDLE other)
{
    HANDLE result;
    THROW_IF_WIN32_BOOL_FALSE(
        DuplicateHandle(GetCurrentProcess(), other,
            GetCurrentProcess(), &result,
            0, FALSE, DUPLICATE_SAME_ACCESS));
    return wil::unique_handle(result);
}

This helper function duplicates a raw HANDLE and returns it in a wil::unique_handle. The duplicate handle has its own lifetime separate from the original. The waiting thread operates on a copy of the handles, so that it is unaffected by changes to the original handles.

std::mutex desiredMutex;
_Guarded_by_(desiredMutex) std::vector<wil::unique_handle> desiredHandles;
_Guarded_by_(desiredMutex) std::vector<std::function<void()>> desiredActions;

The desiredHandles is a vector of handles we want to be waiting for, and the The desiredActions is a parallel vector of things to do for each of those handles.

// auto-reset, initially unsignaled
wil::unique_handle changed(CreateEvent(nullptr, FALSE, FALSE, nullptr));

void waiting_thread()
{
    while (true)
    {
        std::vector<wil::unique_handle> handles;
        std::vector<std::function<void()>> actions;
        {
            std::lock_guard guard(desiredMutex);

            handles.reserve(desiredHandles.size() + 1);
            std::transform(desiredHandles.begin(), desiredHandles.end(),
                std::back_inserter(handles),
                [](auto&& h) { return duplicate_handle(h.get()); });
            // Add the bonus "changed" handle
            handles.emplace_back(duplicate_handle(changed.get()));

            actions = desiredActions;
        }

        auto count = static_cast<DWORD>(handles.size());
                        
        auto result = WaitForMultipleObjects(count,
                        handles.data()->addressof(), FALSE, INFINITE);
        auto index = result - WAIT_OBJECT_0;
        if (index == count - 1) {
            // the list changed. Loop back to update.
            continue;
        } else if (index < count - 1) {
            actions[index]();
        } else {
            // deal with unexpected result
            FAIL_FAST(); // (replace this with your favorite error recovery)
        }
    }
}

The waiting thread makes a copy of the desiredHandles and desiredActions, and adds the changed handle to the end so we will wake up if somebody changes the list. We operate on the copy so that any changes to desiredHandles and desiredActions that occur while we are waiting won’t affect us. Note that the copy in handles is done via Duplicate­Handle so that it operates on a separate set of handles. That way, if another thread closes a handle in desiredHandles, it won’t affect us.

void change_handle_list()
{
    std::lock_guard guard(desiredMutex);
    ⟦ make changes to desiredHandles and desiredActions ⟧
    SetEvent(changed.get());
}

Any time somebody wants to change the list of handles, they take the desiredMutex lock and can proceed to make whatever changes they want. These changes won’t affect the waiting thread because it is operating on duplicate handles. When finished, we set the changed event to wake up the waiting thread so it can pick up the new set of handles.

Right now, the purpose of the changed event is to wake up the blocking call, but we could also use it as a way to know whether we should update our captured handles. This allows us to reuse the handle array if there were no changes.

void waiting_thread()
{
    bool update = true;                        
    std::vector<wil::unique_handle> handles;   
    std::vector<std::function<void()>> actions;

    while (true)
    {
        if (std::exchange(update, false)) {
            std::lock_guard guard(desiredMutex);

            handles.clear();
            handles.reserve(desiredHandles.size() + 1);
            std::transform(desiredHandles.begin(), desiredHandles.end(),
                std::back_inserter(handles),
                [](auto&& h) { return duplicate_handle(h.get()); });
            // Add the bonus "changed" handle
            handles.emplace_back(duplicate_handle(changed.get()));

            actions = desiredActions;
        }

        auto count = static_cast<DWORD>(handles.size());
                        
        auto result = WaitForMultipleObjects(count,
                        handles.data()->get(), FALSE, INFINITE);
        auto index = result - WAIT_OBJECT_0;
        if (index == count - 1) {
            // the list changed. Loop back to update.
            update = true;
            continue;
        } else if (index < count - 1) {
            actions[index]();
        } else {
            // deal with unexpected result
            FAIL_FAST(); // (replace this with your favorite error recovery)
        }
    }
}

In this design, changes to the handle list are asynchronous. They don’t take effect immediately, because the waiting thread might be busy running an action. Instead, they take effect when the waiting thread gets around to making another copy of the desiredHandles vector and call Wait­For­Multiple­Objects again. This could be a problem: You ask to remove a handle, and then clean up the things that the handle depended on. But before the worker thread can process the removal, the handle is signaled. The result is that the worker thread calls your callback after you thought had told it to stop!

Next time, we’ll see what we can do to make the changes synchronous.