MSVC Address Sanitizer adoption in .NET’s CoreCLR

Zachary Johnson

April 12th, 20242 4

Collaboration to improve the reliability and security of .NET, from the perspective of an MSVC Address Sanitizer Developer

Introduction

.NET (on GitHub) is a cross-platform, open-source, and general-purpose development platform with widespread adoption. A core component of .NET is the Core Common Language Runtime (CoreCLR), which provides services such as memory management, exception handling, threading, and interoperability with native code. It is used by millions of developers across multiple platforms, including Windows, Linux, and Mac OSx.

The CoreCLR is implemented in native code, C and C++, and is a complex performance-critical system that interacts with various components, such as the operating system, the native libraries, and managed code. As such, it is susceptible to memory errors such as buffer overflows, use-after-frees, and memory leaks. These errors can cause crashes, undefined behavior, data corruption, security vulnerabilities, and performance degradation. It is crucial to detect and fix them as early as possible in development.

The MSVC Address Sanitizer (ASan) team was able to work with the CoreCLR team to help them consume and integrate ASan into their development workflow. ASan’s promise of zero false positive memory safety error reports increases security confidence in existing and legacy codebases, and CoreCLR was a great candidate for usage. As a bonus, since .NET is cross-platform, it can leverage ASan across Clang, GCC, and now also MSVC. In addition to this work, the native ahead-of-time (NativeAOT) runtime component is also sanitized.

Why were changes required?

Previous work had been done to enable ASan for CoreCLR on GCC, but incompatibilities on Windows with build infrastructure, mixed configurations during testing, and run time issues precluded adoption. However, the effort for GCC was fruitful in finding memory safety errors, such as alloc-dealloc mismatches, buffer overruns, and use-after-scopes.

CoreCLR also contains some unique constraints on Windows that made ASan integration more challenging.

It loads and unloads modules during runtime, which can interfere with ASan’s initialization and finalization.
It uses custom memory allocators and stack unwinding mechanisms, which can conflict with ASan’s tracking and reporting.
It has some code paths containing fast checked casts and tail calls, which originally caused ASan false positives.
It contains a Just-In-Time (JIT) layer that generates machine code, which can be tricky for ASan to navigate.

Work to resolve some issues shared across all platforms was completed previously. However, for CoreCLR on Windows, both build and run time changes were required from the ASan team before adoption could take place.

What changed in MSVC ASan

A significant barrier to entry for several potential consumers of ASan on Windows was the different configurations and consuming debug vs release and dynamic vs statically linked libraries. This problem was addressed by creating a single always-dynamically-linked library that is used as the one source of truth for the application’s memory state. You can read more about the library change in the blog post here. This was also a requirement for ASan adoption in CoreCLR.

Summary of changes:

Annotate allocations that happen prior to when ASan begins tracking the allocations to prevent false positive reports.
Intercept AddVectoredExceptionHandler and RemoveVectoredExceptionHandler on x64 to prevent spurious AccessViolationExceptions. This is necessary due to the shadow memory implementation on x64, which makes use of an on-demand mapping scheme which uses an exception handler to map shadow memory.
Correct an issue regarding ODR (One Definition Rule) violations for global variables of the same size in separate DLLs (Dynamic Link Libraries).
Support for weak symbols to allow the usage of ASan interface functions.
Suppress ASan communication with debuggers to avoid being caught by CoreCLR’s exception handler.

What changed in CoreCLR

After appropriate changes were made from the ASan side, some changes were necessary to avoid build time errors, run time errors, and false positives on the CoreCLR side:

Build infrastructure changes were required to have distinct sanitized and unsanitized component builds to test individual components one at a time, and then later all together. Most of those changes can be seen here.
The __asan_handle_no_return function was inserted into several exception handling code paths to unpoison the stack memory and frames to correctly identify stack-use-after-scope and stack-use-after-return when using a sanitized CoreCLR.
The final test, configuration, and enablement of ASan pull request can be viewed here.

After the changes were made, a weekly pipeline run was created. The overall pipeline stability is still work in progress, but the windows-x64 Checked CoreCLR_RuntimeTests and windows-x64 release NativeAOT steps are the important jobs related to this post. Since the project is so large, and the build and test steps take a significant amount of time already, the decision was to run on a weekly cadence. This prevents hindering code velocity while also providing useful feedback to bisect memory safety errors as they appear. It is important to continue to monitor these results after enabling ASan in projects to not regress memory safety metrics. Much of the work done by both teams was already completed before the release of Visual Studio 2022 17.8, but until the CI infrastructure for .NET could consume the changes in an official release, they were blocked.

Special thanks to Jeremy Koritzinsky and the CoreCLR team for their hard work and collaboration!

Call to action

We’d love for you to download Visual Studio and give the Address Sanitizer a try. As always, we welcome your feedback. We can be reached via the comments below. If you encounter other problems with MSVC or have a suggestion for Visual Studio please let us know through Help > Send Feedback > Report A Problem / Provide a Suggestion in the product, or via Developer Community.