December 16th, 2021

Why does the precise point at which I get a stack overflow exception change from run to run?

Consider this program:

#include <stdio.h>

int maxdepth = 0;

int f()
{
  ++maxdepth;
  return f();
}

int main()
{
  __try {
    f();
  } __except (GetExceptionCode() == STATUS_STACK_OVERFLOW ?
              EXCEPTION_EXECUTE_HANDLER :
              EXCEPTION_CONTINUE_SEARCH) {
    printf("Overflow at %u\n", maxdepth);
  }
}

Make sure to compile this program with optimizations disabled to ensure that the recursive call occupies stack and doesn’t get tail-call-optimized.

When I run this program multiple times, the results are inconsistent.

Overflow at 4882
Overflow at 4877
Overflow at 4879
Overflow at 4884
Overflow at 4877
Overflow at 4883
Overflow at 4882
Overflow at 4881
Overflow at 4879
Overflow at 4882

This program is completely deterministic. Why does the overflow happen at different stack depths? In other words, why does the size of the stack vary?

The variation in the stack size is thanks to Address Space Layout Randomization, or ASLR. The system places the initial stack pointer at a random location in the last page of the stack, thereby randomizing the low-order bits and reducing stack predictability.

It also has a secondary benefit of reducing collisions in caches that are indexed by virtual address. For example, the internal tables used by Wait­On­Address use the address of a stack-allocated object as a key in a hash table to keep track of all the threads that are waiting on an address. Randomizing the initial stack location means that if you have multiple threads running the same code, their stack-allocated objects won’t have similar addresses.

Today, we looked at where the stack starts. Next time, we’ll look at where the stack ends.

Topics
Code

Author

Raymond has been involved in the evolution of Windows for more than 30 years. In 2003, he began a Web site known as The Old New Thing which has grown in popularity far beyond his wildest imagination, a development which still gives him the heebie-jeebies. The Web site spawned a book, coincidentally also titled The Old New Thing (Addison Wesley 2007). He occasionally appears on the Windows Dev Docs Twitter account to tell stories which convey no useful information.

1 comment

Discussion is closed. Login to edit/delete existing comments.

  • Steven Don

    Interesting. I never knew that ASLR would do more than just change the addresses, but could also end up changing the effective size of the stack. I suppose this could lead to random and difficult to debug crashes when stack utilisation is close to the limit. If the sample program had a cut-off at a recursion depth on 4880, it would sometimes crash, sometimes not.