std::string now supports Address Sanitizer

Nicole Mazzuca

When using the Microsoft C++ Standard Library in debug mode (/MTd or /MDd), the library works hard to make sure programmers avoid many access violation bugs. Each container has a custom “wrapped” iterator, which, on every access, checks that it is still valid, isn’t an end iterator, and when doing arithmetic, checks that it’s still in-bound.

However, as soon as you get out of the world of iterators into pointers, these checks can no longer do anything:

int vector_iterators() {
  std::vector<int> v{0, 1, 2, 3, 4, 5};
  return v.begin()[6]; // error at runtime!
}

int vector_data() {
  std::vector<int> v{0, 1, 2, 3, 4, 5};
  return v.data()[6]; // no error reported! undefined behavior!
}

Enter Address Sanitizer (ASan). Add the -fsanitize=address option to your build with either cl or clang, and the compiler will insert checks to make certain that accessed memory is in scope and has not been deallocated.

// compile with -fsanitize=address
int check_stack() {
  int v[] = {0, 1, 2, 3, 4, 5};
  return v[6]; // ASan out-of-bounds access error reported!
}

int check_heap() {
  int *heap_v = new int[6]{0, 1, 2, 3, 4, 5};
  int result = heap_v[6]; // ASan out-of-bounds access error reported!
  delete heap_v;
  return result;
}

int check_vector() {
  std::vector<int> v{0, 1, 2, 3, 4, 5};
  return v.data()[6]; // ASan container-overflow access error reported!
}

Checking stack and raw heap memory has worked since we initially implemented the Address Sanitizer feature. When using containers like std::vector or std::string, it will also make certain you don’t access memory that’s outside the underlying allocation. However, because containers are library code, by default ASan will not prevent you from accessing memory that’s outside the bounds of the container’s capacity, but still inside the bounds of the allocation.

In microsoft/STL#2071, the standard library team added support for ASan container-overflow annotations to std::vector, meaning that our standard library keeps the annotations for std::vector‘s buffer up-to-date manually, using ASan’s __sanitizer_annotate_contiguous_container API. This means that check_vector above correctly errors on a container-overflow error, and we find more bugs in people’s code!

However, std::string is a very different beast to std::vector, due to the Small String Optimization (SSO). In microsoft/STL#2196, we attempted to do an initial implementation, supporting annotations of the SSO buffer as well. However, this had quite a few interesting and unfortunate bugs come out of it, so we needed to disable it again in microsoft/STL#2990.

// compile with -fsanitize=address
char string_old_asan() {
  std::string s = "Hello, world! Let's try address sanitizer!";
  s.reserve(64); // s's heap allocation is now 64 chars wide
  assert(s.size() == 42); // but s still only contains 42 chars
  // before Visual Studio 2022 17.6 Preview 1, this doesn't crash, but reads uninitialized memory
  return s.data()[50]; 
}

However, as of Visual Studio 2022 17.6 Preview 1 (in microsoft/STL#3164), we fixed the bugs and re-enabled the tracking machinery in std::string to keep ASan’s knowledge up to date and correct — meaning that the code sample above suddenly went from silent undefined behavior to very loud undefined behavior! It works with any allocator, including custom allocators, but will check for slightly fewer errors at the boundaries – if you want your custom allocators to fully support ASan checking, you can check out the ASan container-overflow annotations.

One limitation of this checking, in order to avoid the bugs that plagued the original, is that we do not annotate std::string‘s SSO buffer. This means that one can still access out-of-bounds inside the SSO buffer. We’ve left the door open to fixing this in the future, but we wanted to make sure that checking of heap-allocated buffers at least worked.

If you do have issues with container-overflow checking on std::string, you can disable it by passing -D_DISABLE_STRING_ANNOTATION to your compile. If you find bugs in the feature, please report it to either Developer Community or the microsoft/STL GitHub. We can also be reached in the comments below, via the @VisualC twitter account, or via ASanDev@microsoft.com.

Nicole Mazzuca Software Engineer, vcpkg

Follow