⌚ This post was last updated on March 24, 2021 with up-to-date information as of the 16.9 release of Visual Studio.
Special thanks to Jim Radigan and Matthew McGovern for providing the content for this blog post.
Note: This feature is now generally available. To get started, take a look at the official AddressSanitizer for Windows with MSVC documentation.
Last October, we announced AddressSanitizer support for Windows as part of the x86 retail MSVC toolset. This included an IDE experience in Visual Studio with AddressSanitizer error reporting, visualized in dialog boxes anchored to the source code where the error was caught. AddressSanitizer was designed by Google to improve memory safety in programs by identifying violations. It reports precise errors using source lines and symbols. Most importantly, it reports no false positives.
As of Visual Studio 2019 version 16.7, we support both x64 and x86 targets. Both targets also have full support for Debug and fully optimized Release builds. These compilers produce the new code generation and metadata required to interop with the AddressSanitizer runtime. They support this new functionality with all levels of optimization.
To support the IDE and testing workflows, we have included a new vcasan.lib. This small, static library creates metadata the IDE will parse to support error reporting in its sub-panes. This metadata is stored in the Crashdump (.dmp) files produced when a program is terminated by AddressSanitizer when an issue is found. These .dmp files are commonly used to debug causes of unexpected program crashes and can easily be shared with others to replay the program’s behavior in Visual Studio prior to the crash. This functionality allows executables to take advantage of a snapshot process type unique to the Windows operating system.
To fully support those design goals for the Windows platform, we have made changes to enhance the open source runtimes and have more to contribute back to the community. We have fixed bugs in our tools thanks to customers who were early adopters and building large internal systems like Office, all using “–fsanitize=address
.” Finally, we have extended functionalities across the C++ developer stack on Windows. We have more to come in 16.8.
Getting Started with AddressSanitizer for Windows
For more information on how to get started, including installing the necessary components for Visual Studio, enabling build system integration, and running ASan from the command line or IDE, take a look at our previous blog post, AddressSanitizer for Windows with MSVC.
Features available in 16.7
These are AddressSanitizer features (64-bit and 32-bit) that your code can leverage in the 16.7 release:
- stack-use-after-scope
- stack-buffer-overflow
- stack-buffer-underflow
- heap-buffer-overflow (no underflow)
- heap-use-after-free
- calloc-overflow
- dynamic-stack-buffer-overflow (alloca)
- global-overflow (C++ source code)
- new-delete-type-mismatch
- memcpy-param-overlap
- allocation-size-too-big
- invalid-aligned-alloc-alignment
- use-after-poison
- Intra-object-overflow
- Initialization-order-fiasco
- double-free
- alloc-dealloc-mismatch
Features coming in 16.8
The following features are arriving in the 16.8 release:
Global ‘C’ variables We have extended the 16.8 linker to handle global variable scenarios in C code. Recall that in C, a global can be declared many times, and each declaration can be of a different type and size. The only feasible place to allocate C globals is the linker, and it (by convention) just chooses the largest size when there are multiple declarations across .obj files. We will contribute a patch to the upstream LLVM repo so it can target this new capability as well.
__declspec(no_sanitize_address) For certain scenarios, developers might want to opt out of instrumenting entire functions or specific variables. For these uncommon situations, we extended the C/C++ front-ends to annotate variables and functions. We will provide the details of the exact syntax with 16.8 updates.
Automatically link appropriate libraries In 16.7, the developer must explicitly add the correct AddressSanitizer .lib files to the link line when creating a .EXE or .DLL. We have a linker improvement rolling out in 16.8 to remove the need to do this. This will allow the developer to focus only on whether the CRT being targeted is dynamic or is statically linked to the binary being built. Until then, see the Building from CMD section.
Features coming beyond 16.8
Use-after-return This requires code generation that utilizes two stack frames for each function with locals that are tracked by the AddressSanitizer runtime. Running two stacks for one function is complex and just like Clang, a user will have to opt into this feature. It is slow, but it is effective at finding subtle stack corruption bugs impacting reliability or for the security inclined, possible ROP attacks. This is achieved with a simple re-compile.
Building from the command-line
We cover three steps:
- Add flags to the CL command line
- Add libraries to the LINK command line
- Add a directory to your PATH to support errors at runtime
We have been working hard to make sure the AddressSanitizer toolset and runtimes for Windows can be used with all existing compiler and linker flags so that this technology can drop into complex legacy build systems across a broad number of topologies. This includes external developers, large ISVs, and large teams internal to Microsoft (e.g. Office, Windows, and SQL).
In 16.7, a user will have to explicitly add the specific .LIB files needed to link the AddressSanitizer runtime to their existing application. The AddressSanitizer .LIB files needed to build your .EXE or .DLL depend on your choice of CRT:
- Dynamic CRT (/MD)
- Static CRT (/MT)
Note: without either the /MD or /MT flag specified, the static CRT (/MT) is assumed. This has been the default behavior on Windows for years.
CL command line
- set _CL_=
-fsanitize=address /Zi
, or - Manually add
-fsanitize=address /Zi
to all your existing CL command lines
These flags tell the compiler to generate code and layout out stack frames that will interop with the AddressSanitizer runtime. The /Zi
flag will ensures that debug information will be emitted for optimized code. This information ensures the stack-walker can print stack frames with function names and source line numbers when reporting an error in your code.
LINK.exe command line
Depending on your version of Visual Studio, you may need to perform an additional step to enable the Address Sanitizer experience with the MSVC linker for command line builds.
16.9 Preview 3 and later
16.9 Preview 2 and earlier
The AddressSanitizer runtimes will “hook” many entry points in the VC++ runtimes. For example, the AddressSanitizer runtimes need to direct malloc and free temporarily to the AddressSanitizer runtime to track heap allocations and return through the original CRT bodies. This means the CRT entry points determine which AddressSanitizer .LIB files you need to explicitly link with the binary you are building. Assume your installation location is cached in an environment variable MyVS with the following:
set MyVS= C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\VC\Tools\MSVC\14.27.29109
Building an x86 .EXE linked with dynamic CRT:
set _LINK_= /debug -incremental:no
/wholearchive:%MyVS%\lib\x86\clang_rt.asan_dynamic-i386.lib
/wholearchive:%MyVS%\lib\x86\clang_rt.asan_dynamic_runtime_thunk-i386.lib
Building an x86 .EXE linked with static CRT:
set _LINK_= /debug -incremental:no
/wholearchive:%MyVS%\lib\x86\clang_rt.asan-i386.lib
/wholearchive:%MyVS%\lib\x86\clang_rt.asan_cxx-i386.lib
Building an X64 .EXE linked with dynamic CRT:
set _LINK_= /debug -incremental:no
/wholearchive: %MyVS%\lib\x64\clang_rt.asan_dynamic-x86_64.lib
/wholearchive: %MyVS%\lib\x64\clang_rt.asan_dynamic_runtime_thunk-x86_64.lib
Building an x64 .EXE linked with static CRT:
set _LINK_= /debug -incremental:no
/wholearchive:%MyVS%\lib\x64\clang_rt.asan-x86_64.lib
/wholearchive:%MyVS%\lib\x64\clang_rt.asan_cxx-x86_64.lib
Building DLLs linked with dynamic CRT:
Link both the EXE and DLL with:
set _LINK_= /debug -incremental:no /wholearchive:%MyVS%\lib\{arch}\clang_rt.asan_dynamic-{arch}.lib
/wholearchive:%MyVS%\lib\{arch}\clang_rt.asan_dynamic_runtime_thunk-{arch}.lib
Building DLLs linked with static CRT:
Link the EXE with:
set _LINK_= /debug -incremental:no
/wholearchive:%MyVS%\lib\{arch}\clang_rt.asan-{arch}.lib
/wholearchive:%MyVS%\lib\{arch}\clang_rt.asan_cxx-{arch}.lib
Link the DLL with:
set _LINK_= /debug -incremental:no
/wholearchive:%MyVS%\lib\{arch}\clang_rt.asan_dll_thunk-{arch}.lib
Symbolizer – running your application
When running an application compiled with –fsanitize=address
, make sure the AddressSanitizer runtime can find its “symbolizer.” The runtime calls out to llvm-symbolizer.exe. This will walk the stack in your failing program and print active function names and line numbers as part of a detailed diagnostic error message.
The symbolizer is in the default directory with the compiler and linker. So, when running:
- On x86, set
path=%path%; %MyVS%\bin\Hostx64\x86
- On X64, set
path=%path%; %MyVS%\bin\Hostx64\x64
Vcasan.lib – experimental
We have added a new static library that is automatically used when building AddressSanitizer from the IDE and project system. For command line builds, depending on your version of Visual Studio, an additional step may be required.
16.9 Preview 3 and later
Starting with 16.9 Preview 3, vcasan.lib is automatically linked for you for command line builds. You should not need to do any additional work to use it,
16.9 Preview 2 and earlier
For older versions of Visual Studio, if you are building from the command line, you must manually link this library to the binary you are building. The library can be used to capture failures to disk for offline viewing in Visual Studio. This can help with automated test systems and cloud-based workflows that use the AddressSanitizer.
For 16.7, when you have linked vcasan.lib to your executable, you can set an environment variable:
set ASAN_SAVE_DUMPS=”MyFileName.dmpx”
This will save a snapshot file when an error is caught by the AddressSanitizer. The meta-data that is saved in the dump file is parsed by the new Visual Studio IDE. You can set this variable on a per test basis and store these binary artifacts and then view these in the IDE with proper source indexing.
Known issues
Alignment
There is a known issue with false positives being reported for locals that are manually aligned:
_declspec(align(n)
#pragma align(n)
#pragma pack(n)
To provide some insights, consider __declspec(align(32))
. The size of the object is used to calculate offsets in arrays and when you use pointers, so sizeof(x)
must always be a multiple of the alignment value. In this case, 1 x 32.
But if you have __declspec(align(32)) struct aType {int a[12]; };
, then the size would be 2 x 32 = 64, since sizeof(a)
is 12 x 4 = 48. If we change it to align to 4, 8 or 16, it would be 48.
The code the compiler needs to generate (for all alignment scenarios) can get complex, and this is an area we are finishing up with extended stress testing. In the meantime, bugs in this area will result in false positives.
Debugging – Exceptions (on 16.9 Preview 3 and earlier)
On version 16.9 Preview 3 or earlier, see the instructions below. For Preview 4 onward, these actions are not necessary.
Once you have built an executable that is compiled with -fsanitize=address
, you may want to run the application under the debugger. The Address Sanitizer runtime that was just linked to your application will begin to page in the virtual space. Rather than statically reserving a large amount of possibly unused memory, the virtual space is expanded through exceptions.
You can use the debugger from the command line as follows:
devenv.exe /debug my.exe
Your debug session will experience these exceptions. You need to disable the Win32 exception for breaking when there’s an access violation.
In Visual Studio, use Debug > Windows > Exception Settings to open the Exception Settings window and uncheck the 0xc0000005
checkbox seen below:
Debugging – Source line numbers (on 16.9 Preview 3 and earlier)
On version 16.9 Preview 3 or earlier, see the instructions below. For Preview 4 onward, these actions are not necessary.
There is a known issue with keeping line number correct for the synthetic code the compiler injects for instrumenting your application. When single stepping, the cursor might jump sporadically then return to your source code. This is a bug that’s being fixed now.
Other limitations
Please note the following additional limitations in the experience:
- Native/managed interoperability is not fully supported at this time. Your C/C++ binaries linked with the AddressSanitizer runtime may encounter issues with heap allocations in the different domains.
- When compiling with optimization and inlining, the line number and column information can be off. Under
/O2
and above, we aggressively transform programs and lose tight correlation with linear source code. If needed, you can always compile/Od
to get correct source mapping. - Optimization –
/Od
vs./O2
vs./LTCG
. The various levels of optimization attempt to keep as many operands in registers as possible while inlining without excessive code size bloat. The compiler only instruments memory references when–fsanitize=address
is added to a command line. If you are compiling the same code/Od
, then/O2
, and then/O2 /LTCG
, loads and stores may have been optimized away or moved from different functions. If the optimizer was successful, then what’s reported using/Od
may not be seen when compiling/O2
. Similarly, what’s reported/O2
may not be seen when compiling/O2 /LTCG
. - Loading a DLL built
–fsanitize=address
and running with an EXE not compiled–fsanitize=address
is unsupported. Even calling LoadLibary from main would result in false positives. - Just calling LoadLibrary without linking the import library and runtime for the AddressSanitizer runtime (see Building from CMD above) will cause false positive bugs.
Acknowledgements
We want to thank Kostya Serebreyany, Reid Kleckner, Vitaly Buka, Martin Storsjö, and the rest of the LLVM developer community for their input and continued work on LLVM and all the other sanitizers.
If you want to learn more about the AddressSanitizer, Google has published an overview about the algorithm and the implementation. Their documentation also details the various AddressSanitizer runtime options which are selectable via the ASAN_OPTIONS environment variable. These function for both CLANG and MSVC implementations of the AddressSanitizer because they share a common set of runtime libraries. Also, check out the original paper on AddressSanitizer.
We want your feedback!
Your feedback is key for us to deliver a great experience in Visual Studio and the MSVC toolset. We’d love for you to try out the latest preview version of Visual Studio 2019 and let us know how it’s working for you, either in the comments below or via email. If you encounter problems with the experience or have suggestions for improvement, please Report A Problem or reach out via Developer Community. You can also find us on Twitter @VisualC.
APPENDIX – Advanced users
The rest of this post contains additional details for more advanced usage scenarios for ASan on Windows to help developers understand the level of support available.
Windows Heap interception functionality
The ASan runtime intercepts a specific set of allocation and memory management functions. HeapAlloc
, GlobalAlloc
, LocalAlloc
, and RtlAllocateHeap
plus their corresponding Free
, Size
, and ReAlloc
functions are supported. Users can enable the Windows runtime library interception routines by adding windows_hook_rtl_allocators=true
to the ASAN_OPTIONS environment variable before executing their program.
As a usability note, we recommend setting ASAN_OPTIONS=windows_hook_rtl_allocators=true
at the user scope before starting Visual Studio to ensure the environment option persists through other program configuration changes.
Not all heap feature flags are supported at this time:
HEAP_ZERO_MEMORY
is supported for all Windows heap Alloc, Free, and ReAllocHEAP_NO_SERIALIZE
is not supported for any Windows heap functionsHEAP_GENERATE_EXCEPTIONS
is not supported for any Windows heap functionsHEAP_REALLOC_IN_PLACE_ONLY
is not supported for any Windows heap ReAlloc
In cases where an allocation flag is not supported, the AddressSanitizer runtime will transfer the allocation to the original Windows heap allocator to preserve program execution. This can result in false negatives if memory corruption occurs within code that makes use of these unsupported heap flags in the Windows heap functions.
Overloading of new/delete
The AddressSanitizer runtime overloads new and delete in the asan_cxx
runtime for C++. If the user also overloads these library functions, it is possible to miss bug reports because the runtime no longer intercepts and tracks allocations.
Failing to overload the full set of new/delete/new[]/delete[]
can also cause inconsistencies when the AddressSanitizer runtime is loaded. This scenario can lead to false positive bugs. If your code cannot avoid overloading global new/delete operators, you may need to delay using the AddressSanitizer until a runtime solution is ready.
Custom allocators
A user can enlighten their allocator with manual poisoning/unpoisoning of shadow memory. For more details on this process, please review the documentation. Be sure to #include sanitizers/asan_interface.h
to get access to the user API for reading and writing shadow bytes.
Debug libraries
- The Windows debug runtimes have additional asserts and preprocessor definitions enabled when they are compiled. These can cause extra debug breakpoints and assertions to be thrown even when C++ binaries are built without
–fsanitize=address
. Users may continue past these breakpoints, but please report any runtime assertions you encounter while testing ASan. - Compiling with the static debug runtimes might require the use of
/force:multiple
at the linking phase. While malloc and free are designed to be overloaded by the user,malloc_dbg
andfree_dbg
are not designed to be overloaded. The linker will throw a ‘multiple symbol definition’ error. The ‘/force:multiple’ switch will force the linker to ignore this warning and use the AddressSanitizer function definitions. There will be some warning output which will detail which version of the function was chosen. This output can be used to verify the ASan version of the[malloc|free|msize|...]_dbg
set of CRT memory functions has been correctly selected.
X64 SUPPORT
First-chance AV exceptions
The AddressSanitizer runtime on x64 Windows, makes use of an on-demand mapping scheme which uses an exception handler to map shadow memory. This is present in the open source version and our Visual Studio provided version currently. When debugging, these exceptions will appear as first-chance access violation exceptions with code 0xC0000005. There will be a significant number of them, but these first-chance exceptions can be safely ignored or disabled. Windbg users can use sxd av
to ignore first-chance access violation exceptions. Visual Studio users can opt to ignore the exception type when prompted on the first occurrence. The runtime is designed to catch any actual access violations and report them before they occur.
Note: x86 does not use this memory model. The x86 version simply maps 200 MB of process memory at 0x30000000 during early program initialization for use as the shadow memory area.
Global reporting and x64 debug assertion
The x86->x64 cross compiler can cause issues with global variables currently. This issue will cause an assertion in the AddressSanitizer runtime while the program is initializing. If you experience this error, set PreferredToolArchitecture=x64
to force Visual Studio to use the x64 native toolset.
Issue suppression
Turning off AddressSanitizer instrumentation using _declspec(no_sanitize_address)
is not available in 16.7 currently. The runtime has a few methods of dealing with known or “on-purpose” memory violations. Suppression files can skip reporting on a bug in a single function. Details can be found at https://clang.llvm.org/docs/AddressSanitizer.html#issue-suppression.
Hi,
Is this not mature on Debug X64?
I create a simple console application for testing purposes, I list some issues I encounter as below.
1.
Issue: F5 Start Debugging -> Exception by access violation before the application enter main function.
Workaround: I use Sleep(....) and attach the process for debugging instead.
2.
Issue: False Alarm makes the application abort.
Example: Just copy below code and run, it will crash at
m_locator->ConnectServer but the HRESULT is...
Is there a way to tell it not to check external code? Our project uses MFC, and the first exception is this:
HEAP[NG-DART.exe]: Invalid address specified to RtlValidateHeap( 02B50000, 05B1A1E0 )
NG-DART.exe has triggered a breakpoint.
C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\atlmfc\src\mfc\winhand.cpp Line 56:
CWinApp* pApp = AfxGetApp();
_AFX_THREAD_STATE* pThreadState = _afxThreadState.GetDataNA();
if( pThreadState != NULL )
{
// restore safety pool after temp objects destroyed
if (pApp != NULL &&
(pThreadState->m_pSafetyPoolBuffer == NULL ||
Line 56: _msize(pThreadState->m_pSafetyPoolBuffer)...
Hi Scott,
I’m unaware of any way to tell ASAN to ignore external code.
However, I can tell you that ignoring that breakpoint is almost certainly fine.
I’ll take a look at what it would take to avoid throwing an exception in this situation.
Curtis
Is there a way to tell the LLVM Symbolizer where to look for PDBs? I’ve built a thirdparty library with ASAN on our buildserver and when the symbolizer tries to fetch the PDBs for that library it tries to use PDBs from the build directory of that library, which doesn’t exist on my machine.
Is there a way to detect memory leak using this?
Yes, some. For example, if you don’t use a virtual destructor you might see an error message that says something like: “new and delete mismatch. You allocated 12 bytes but freed 8”.
-Jim
Hello, has anybody tried to use Asan with Qt on Windows?
I am trying to use Asan with Qt 5.15.0 C++ project compiled with MSVC 2019 16.7.30406. It is a simple "Hello world" application.
I specified QMAKE_CXXFLAGS += -fsanitize=address
CONFIG(debug, debug|release): QMAKE_LFLAGS += /wholearchive:clang_rt.asan_dbg_dynamic-x86_64.lib
CONFIG(debug, debug|release): QMAKE_LFLAGS += /wholearchive:clang_rt.asan_dbg_dynamic_runtime_thunk-x86_64.lib
And I disabled first chance exceptions in CDB debuugger.
Application seems to work, but when I run it from Qt Creator tool and then close - it crashes on...
Hello,
This project seems great !
I just try to set it up and running for Qt 5.15.0 - x64
Test made in release mode with the build-in sample - simplebrowser.pro
To make it work, I just define thoses in my .pro.
<code>
I also add the bin path to the %path%
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\Hostx86\x64
Inside QtCreator, output result
<code>
love QtCreator, but too basic to really debug advanced problems.
ok I give a try in Microsoft Visual Studio Community...
Thanks Renan and Boris. Sorry there were problems with Qt.
We are going to build Qt 5.15.0 – x64 with -fsanitize=address internally and test the scenarios that have been described above.
The failure when exiting, seem like memory allocation from another DLL that was being used in the application when it was tearing down. That might explain why it worked from the command line, but not within the Qt Creator tool. Will have to...
Hello Jim,
Did you give a try to Qt ?
I still got stars into my eyes since I read Augustin post.
AddressSanitizer is a must have for any project !
Renan
Hi again Renan,
Have you had a chance to try this out with 16.8.0? I can’t seem to repro what you are describing.
Curtis
Hi Renan,
I’m in the process of looking at this!
Curtis
this comment has been deleted.
Hi,
First try on one of our larger DLL, dynamic crt.
suggested CL LINK options are added to vcxproj. (rumtime checks turned off)
I got this
1>LINK : fatal error LNK1181: cannot open input file 'Files.obj'
1>Done building project "KernelAPI.vcxproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========
We don't have Files.c or .cpp or .* source in this DLL (or any other dll or lib in this build tree),...
Is there a sensible way to remove the problems with mixing executables and DLLs built with and without ASAN? This precludes useful scenarios like plugins being built with ASAN being used by a host application built without ASAN. (eg. Excel plugin).
In the Building from Command Line section (https://devblogs.microsoft.com/cppblog/asan-for-windows-x64-and-debug-build-support/#building-from-the-command-line), are we forgetting to mention use of `/MDd` and `/MTd`? It's kinda of implied that this is supported from the comments about `malloc_dbg()`.
In my testing, a non-ASAN built executable seems to be unable to successfully call LoadLibrary() on a DLL built with ASAN. Is that expected?
EDIT: It seems that there's some basic problem with static initializers and LoadLibrary(), I'm seeing bad memory during static initialization even from...
Hi,
Unfortunately a non-ASAN built executable can NOT call LoadLibrary() on a DLL built with ASAN. The core issue is that the AddressSanitizer runtime is tracking memory and the non-ASAN executable might have done something like HeapAlloc. We call this out in the blog. This is an existing limitation of the runtime and it thwarts scenarios like the Excel plugin that you aptly point out in your next post. We are considering work to deal...
Is Asan compatible with CMake builds using cl.exe? If so, Are there any extra steps to follow?
Yes it is, steps are described in this blog post under the “Turning on ASan for Windows CMake projects” section: https://devblogs.microsoft.com/cppblog/addresssanitizer-asan-for-windows-with-msvc/
Actually trying to use this with a CMake build that already supports this in for gcc/clang and XCode, VS needs to improve the user experience (this is not the VS CMake integration, but the CMake VisualStudio generator).
If the linker would accept /fsanitize=address and add the right ASan runtime libraries (and whatever else it needs) for you, then it might be usable. From what I read, and my own attempts to get it...
Typo? 0xC000000D - they're 0xC0000005.
There are many of them too, so it's best to disable VS's 0xc0000005 Break When Thrown exception setting.
Also, VS no longer reports them as First Chance exceptions. It'd be good if it stuck with that terminology as it's one that people recognise.
ASAN is also affecting debugging adversely currently. Even in a full debug non-optimized build, stepping through code occasionally appears to jump back to main() (I'm...
Hi David, thank you for the feedback and for catching the issue with that address. I updated the blog post. For the jumping issue, can you also file a bug on Developer Community? https://developercommunity.visualstudio.com/index.html
The engineering team is aware of that particular issue but this will allow us to notify you when it is addressed or for the team to ask you follow-up questions about your experience.
Already reported