September 18th, 2013

SubtractRect doesn't always give you the exact difference

The Subtract­Rect function takes a source rectangle and subtracts out the portion which intersects a second rectangle, returning the result in a third rectangle. But wait a second, the result of subtracting one rectangle from another need not be another rectangle. It might be an L-shape, or it might be a rectangle with a rectangular hole. How does this map back to a rectangle?

The documentation for Subtract­Rect says that the function performs the subtraction when they “intersect completely in either the x- or y-direction.” But I prefer to think of it as the alternate formulation offered in the documentation: “In other words, the resulting rectangle is the bounding box of the geometric difference.”

I was reminded of this subject when I saw some code that tried to do rectangle manipulation like this:

// Clip rcA to be completely inside rcB.
RECT rcSub;
// rcSub = the part of rcA that stick out beyond rcB
if (SubtractRect(&rcSub, &rcA, &rcB)) {
    // Remove that part from rcA
    SubtractRect(&rcA, &rcA, &rcSub);
}

If the rectangle rcA extends beyond rcB in more than one direction, then the geometric difference will not be rectangular, and the result of Subtract­Rect will be expanded to the bounding box of the difference, which means that it will return rcA again. And then the second line will subtract it all out, leaving the rectangle empty.

Oops.

What they really wanted was

// Clip rcA to be completely inside rcB.
IntersectRect(&rcA, &rcA, &rcB);
Topics
Code

Author

Raymond has been involved in the evolution of Windows for more than 30 years. In 2003, he began a Web site known as The Old New Thing which has grown in popularity far beyond his wildest imagination, a development which still gives him the heebie-jeebies. The Web site spawned a book, coincidentally also titled The Old New Thing (Addison Wesley 2007). He occasionally appears on the Windows Dev Docs Twitter account to tell stories which convey no useful information.

0 comments

Discussion are closed.