The Old New Thing

Mismatching scalar and vector new and delete

In a previous entry I alluded to the problems that can occur if you mismatch scalar "new" with vector "delete[]" or vice versa. There is a nice description of C++ memory management in C++ Gotchas: Avoiding Common Problems in Coding and Design on www.informit.com, and I encourage you to read at least the section titled Failure to ...

Integer overflow in the new[] operator

Integer overflows are becoming a new security attack vector. Mike Howard's article discusses some of the ways you can protect yourself against integer overflow attacks. One attack vector he neglects to mention is integer overflow in the new[] operator. This operator performs an implicit multiplication that is unchecked: If you study the ...

Another reason not to do anything scary in your DllMain: Inadvertent deadlock

Your DllMain function runs inside the loader lock, one of the few times the OS lets you run code while one of its internal locks is held. This means that you must be extra careful not to violate a lock hierarchy in your DllMain; otherwise, you are asking for a deadlock. (You do have a lock hierarchy in your DLL, right?) The loader lock ...

Some reasons not to do anything scary in your DllMain

As everybody knows by now, you're not supposed to do anything even remotely interesting in your DllMain function. Oleg Lvovitch has written two very good articles about this, one about how things work, and one about what goes wrong when they don't work. Here's another reason not to do anything remotely interesting in your DllMain: It's ...

The hollow brush

What is the hollow brush for? The hollow brush is a brush that doesn't do anything. You can use it when you're forced to use a brush but you don't want to. As one example, you can use it as your class brush. Then when your program stops responding and Windows decide to do the "white flash" (see yesterday's entry), it grabs the hollow brush ...

ia64 – misdeclaring near and far data

As I mentioned yesterday, the ia64 is a very demanding architecture. Today I'll discuss another way that lying to the compiler will come back and bite you. The ia64 does not have an absolute addressing mode. Instead, you access your global variables through the r1 register, nicknamed "gp" (global pointer). This register always points to your ...

Uninitialized garbage on ia64 can be deadly

On Friday, we talked about some of the bad things that can happen if you call a function with the wrong signature. The ia64 introduces yet another possible bad consequence of a mismatched function signature which you may have thought was harmless. The CreateThread function accepts a LPTHREAD_START_ROUTINE, which has the function signature ...

How can a program survive a corrupted stack?

Continuing from yesterday: The x86 architecture traditionally uses the EBP register to establish a stack frame. A typical function prologue goes like this: This establishes a stack frame that looks like this, for, say, a __stdcall function that takes two parameters. Parameters can be accessed with positive offsets from EBP; for example...

What can go wrong when you mismatch the calling convention?

Believe it or not, calling conventions is one of the things that programs frequently get wrong. The compiler yells at you when you mismatch a calling convention, but lazy programmers will just stick a cast in there to get the compiler to "shut up already". And then Windows is stuck having to support your buggy code forever. The window ...