Address Sanitizer for MSVC Now Generally Available
Augustin Popa
This post was last updated on March 10th, 2021.
Special thanks to Aaron Gorenstein who provided most of the content for this blog post. Special thanks as well to Kevin Cadieux and Jim Radigan who also made contributions.
You can check out a demo on this feature in the latest Visual Studio Toolbox episode on YouTube: Visual Studio Toolbox Live – What’s New in Visual Studio 2019 version 16.9? – YouTube
To get started with this experience, check out our documentation.
—
We’re thrilled to announce that as of Visual Studio 2019 version 16.9, the C++ Address Sanitizer (ASan) for MSVC experience is fully supported. Thanks to all who tried it out while it was experimental in earlier versions of the IDE and filed issues to help make this release all-the-better!
The Address Sanitizer
The Address Sanitizer (ASan) is an immensely valuable technology for finding and fixing memory bugs. Here’s a quick illustration:
It was originally designed and implemented for the LLVM/Clang compiler. Starting in version 16.1, we brought the technology to Visual Studio 2019 for targeting Linux, then added MSVC support in version 16.4. Now with version 16.9, we have altogether resolved over 50 customer-reported bugs and improved the user experience to the point where we are ready to announce that we are moving the ASan for Windows experience out of experimental state. We consider the feature stable and ready to use for production environments. When we last blogged about the AddressSanitizer for Windows experience with version 16.7, we had announced x64 and Debug target support. Since then, we simplified the compilation model to match LLVM by automatically linking runtime libraries when the /fsanitize=address compiler flag is specified. Now, most users will just need to add that single flag to get ASan, with no additional flags or linker options needed. We also polished the debugging experience by cleaning up runtime-generated exceptions and fixing line number mismatches.
ASan helps catch a variety of memory issues, all of which can have real impact in running your programs. These issues range from the “classics” like use-after-free, to subtler issues like the use of a local variable after it has left scope. We’ve provided documentation on how to get started with ASan with Visual Studio in a variety of build environments (MSBuild, CMake, and the command line), any of which require just a few small changes. Read on to learn more of what ASan is and how it can help you in developing your software.
The Value
We’re grateful that the core ASan technologies and benefits are well-established with LLVM/Clang. It’s been exciting bringing those benefits to MSVC customers and see just how it can help find and investigate a great many bugs. Some examples:
- To validate our ASan feature, we ran our test suite with the ASan option turned on. The test suite includes Electron, Qt Creator, Eigen, FLAC, postgresql, OpenSSL, and around 30 other projects.
- We built the MSVC compiler itself with ASan and ran some key tests. This is helping us identify bugs in our C++ toolchain. Furthermore, it is saving us time identifying issues when we investigate MSVC bugs reported by the community.
- The Windows team is fuzzing their codebase with ASan and OneFuzz and their team has been instrumental in identifying and fixing issues in the ASan for Windows experience.
- We are expanding efforts to use ASan internally at Microsoft, including in the Windows and Office organization.
We will be saying more about the topics above in some future blog posts. We’ve also been contributing our changes to the ASan components shared between MSVC and Clang upstream to the shared OSS repo. The latest of our PRs has been submitted for review, adding support for tracking allocations made through low-level Windows APIs. We expect with future fixes and enhancements there will be more. Our north star is to always contribute to, and use, the same OSS ASan components.
Moreover, with version 16.9 we have full Visual Studio IDE integration with ASan. Now you can obtain the core value of the fundamental ASan technology in the first-class Visual Studio debugging experience on Windows. In the opening picture you see how Visual Studio points right to the ASan-detected bug, complete with explanatory pop-up, call stack, and all the other debugging information you’d expect. We’ve documented more of the details of the debugger integration, and the text-based ASan report is still available in the Output window for advanced users.
Moving beyond the local dev loop, we also added the ability to save ASan crash dumps, thereby enabling whole new cloud and distributed testing scenarios. A deep dive on this scenario can be found in our documentation.
Getting Started
If you’re eager to jump in and try ASan for yourself, check out our documentation.
The Journey, and Thanks!
Bringing ASan to Visual Studio as a first-class experience took time. Of course, we are grateful for the LLVM/Clang team that created and continues to develop ASan, and we feel exceptionally grateful to the many customers who took our invitation to try out ASan while it was experimental and helped us improve its quality. False positives, those where ASan reports a memory bug that is not truly a memory bug, we treat with highest priority and we strain to fix those as soon as possible. False negatives, where ASan should have reported an issue but didn’t, are intrinsically harder to find but fortunately less disruptive; to maximize the value of ASan we also strive to fix those quickly. We have been heads-down working to address the last, recently-reported issues. While we of course hope that the ASan integration quality satisfies your requirements, we have documented an “escape hatch” to selectively turn off ASan in the rare case you do hit a bug.
We see ASan as just the start of bringing more sanitizer-and-related technology to Visual Studio. If you have thoughts on the future of undefined-behavior-, memory-, thread-, or other-sanitizers, please share them as a suggestion on Developer Community! If you suspect you’ve hit an issue or bug, please also don’t hesitate to open a ticket on Developer Community!
Thanks for reading!
Light
Dark
22 comments
—————————
RegressionTest.exe – System Error
—————————
The code execution cannot proceed because clang_rt.asan_dbg_dynamic-x86_64.dll was not found. Reinstalling the program may fix this problem.
—————————
OK
—————————
Hi Peter, the binaries should already be on the path in any Developer Command Prompt, or from terminals spawned from the Visual Studio IDE. If you need to redistribute the binaries for test-only purposes, they are next to the compiler bits (next to cl.exe) in the VS install folder.
Hi, VS restarted, in newly opened Visual Studio 2019 Developer Command Prompt v16.9.0. Console mytest exe tries to use one asan enabled rebuilt DLL for checking. The error msg remains.
clang_rt.asan_dbg_dynamic-x86_64.dll is in
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\bin\Hostx64\x64
and
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\bin\Hostx86\x64
The PC was not restarted yet.
Hi Eric,
After upgrading VS 2019 to the latest, I get the same dll not found error as Peter, running from the IDE when trying to use the sanitizer. I need to set the PATH in the Debugging properties to point to where that dll is, in order to be able to make it work.
After reboot:
My VS -> Debugging -> Environment:
PATH=C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\bin\HostX86\x86;%PATH%
ASAN_SYMBOLIZER_PATH=C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\bin\HostX86\x86
ASAN_OPTIONS=
F5
No error msg dlg but:
Exception thrown at 0x00007FF8A3B53738 (ntdll.dll) in RegressionTest.exe: 0xE0736171: Access violation writing location 0x0000040C58D60000.
> ntdll.dll!memset() Unknown
clang_rt.asan_dbg_dynamic-x86_64.dll!00007fffe7f20d0e() Unknown
clang_rt.asan_dbg_dynamic-x86_64.dll!00007fffe7efd9a7() Unknown
clang_rt.asan_dbg_dynamic-x86_64.dll!00007fffe7efd17e() Unknown
clang_rt.asan_dbg_dynamic-x86_64.dll!00007fffe7f25e62() Unknown
ucrtbased.dll!00007ff805774299() Unknown
clang_rt.asan_dbg_dynamic-x86_64.dll!00007fffe7f30a75() Unknown
clang_rt.asan_dbg_dynamic-x86_64.dll!00007fffe7f30967() Unknown
clang_rt.asan_dbg_dynamic-x86_64.dll!00007fffe7f30c95() Unknown
clang_rt.asan_dbg_dynamic-x86_64.dll!00007fffe7f30e91() Unknown
ntdll.dll!LdrpCallInitRoutine() Unknown
ntdll.dll!LdrpInitializeNode() Unknown
ntdll.dll!LdrpInitializeGraphRecurse() Unknown
ntdll.dll!LdrpInitializeGraphRecurse() Unknown
ntdll.dll!LdrpInitializeGraphRecurse() Unknown
ntdll.dll!LdrpInitializeGraphRecurse() Unknown
ntdll.dll!LdrpInitializeProcess() Unknown
ntdll.dll!_LdrpInitialize() Unknown
ntdll.dll!LdrpInitialize() Unknown
ntdll.dll!LdrInitializeThunk() Unknown
Different exception here.
Exception thrown at 0x00007FFD29277034 (ntdll.dll) in xxxx_ud64.exe: 0xC000001D: Illegal Instruction.
Unhandled exception at 0x00007FFD29277034 (ntdll.dll) in xxxx_ud64.exe: 0xC000001D: Illegal Instruction.
> ntdll.dll!wcslen() Unknown
clang_rt.asan_dbg_dynamic-x86_64.dll!00007ffcad2f7099() Unknown
ucrtbased.dll!00007ffcacd17b70() Unknown
mfc140ud.dll!00007ffcaacb5e73() Unknown
mfc140ud.dll!00007ffcaacb5c07() Unknown
mfc140ud.dll!00007ffcaaf0336c() Unknown
mfc140ud.dll!00007ffcaaf0d2a8() Unknown
mfc140ud.dll!00007ffcaaf0d3e1() Unknown
ntdll.dll!LdrpCallInitRoutine() Unknown
ntdll.dll!LdrpInitializeNode() Unknown
ntdll.dll!LdrpInitializeGraphRecurse() Unknown
ntdll.dll!LdrpInitializeGraphRecurse() Unknown
ntdll.dll!LdrpInitializeGraphRecurse() Unknown
ntdll.dll!LdrpInitializeProcess() Unknown
ntdll.dll!_LdrpInitialize() Unknown
ntdll.dll!LdrpInitializeInternal() Unknown
ntdll.dll!LdrInitializeThunk() Unknown
I am running on a Windows 10 21322.rs_prerelase.210220-1651 on an AMD Ryzen 9 3950X.
The project is compiled in 64 bit but linked with VS2017 DLLs. Not sure if that is relevant or not.
Actually the first exception I am getting has a very similar callstack (after loading the symbols that is a bit more informative)
ntdll.dll!memset() Unknown
[External Code]
ucrtbased.dll!_initterm(void(*)() * first, void(*)() * last) Line 22 C++
> clang_rt.asan_dbg_dynamic-x86_64.dll!dllmain_crt_process_attach() Unknown
clang_rt.asan_dbg_dynamic-x86_64.dll!dllmain_crt_dispatch() Unknown
clang_rt.asan_dbg_dynamic-x86_64.dll!dllmain_dispatch() Unknown
clang_rt.asan_dbg_dynamic-x86_64.dll!_DllMainCRTStartup() Unknown
ntdll.dll!LdrpCallInitRoutine() Unknown
ntdll.dll!LdrpInitializeNode() Unknown
ntdll.dll!LdrpInitializeGraphRecurse() Unknown
ntdll.dll!LdrpInitializeGraphRecurse() Unknown
ntdll.dll!LdrpInitializeProcess() Unknown
ntdll.dll!_LdrpInitialize() Unknown
ntdll.dll!LdrpInitializeInternal() Unknown
ntdll.dll!LdrInitializeThunk() Unknown
Who else thought this was about address sanitisation? The process of looking at a delivery point and matching it to the postal service one to fix errors or remove fakes.
Anyone who is interested in LeakSanitizer support please vote for https://developercommunity2.visualstudio.com/t/support-LeakSanitizer/826620 !
I just updated to 16.9.1, and the Address Sanitizer option is present in the project settings for Win32 configurations, but when I switch to x64 the option disappears entirely.
Debug vs Release makes no difference.
Is this a bug? Or is there some other reason why it would be hidden in x64?
This would definitely be a bug since ASan should work for both x64 and x86. I am unable to reproduce this though on my machine. Could you file a bug through Help > Send Feedback > Report a Problem in the IDE and include a screenshot of your project properties window on the x64 view? Are you on the Configuration Properties > C/C++ > General page?
Yes, the C/C++ > General page. I can see the ASan option there for Win32, but switching to x64 makes it disappear.
I have submitted a bug report as you suggested.
As an experiment, I tried replicating this on a new solution. This bug actually did not manifest on a simple Hello World app, but my work solution with many projects still has the bug. I wonder if there is some setting somewhere that is interfering…
Awesome stuff folks! One question though: What is the recommended way of setting up a project to be able to reference the sanitizer/asan_interface.h header file to use the ASAN_POISON_MEMORY_REGION/ASAN_UNPOISON_MEMORY_REGION macros.
The file is shipped with msvc but newly created projects don’t seem to have that as an include path.
Hi Roger, you can follow the instructions here: https://docs.microsoft.com/en-us/cpp/sanitizers/asan-runtime?view=msvc-160#poisoning along with some examples:
#include “sanitizer/asan_interface.h”
Thanks,
Eric
Well the crux of my question is that the include paths aren’t setup correctly. I get the following error: cannot open source file “sanitizer/asan_interface.h”
Does a particular include path need to be manually defined in the solution/cmake files first? If so, what would that path be?
The file is shipped 3 times but neither of these paths are part of a project’s typical include search path:
[.\2019\Preview\VC\Tools\Llvm\lib\clang\11.0.0\include\sanitizer]
[.\2019\Preview\VC\Tools\Llvm\x64\lib\clang\11.0.0\include\sanitizer]
[.\2019\Preview\VC\Tools\MSVC\14.29.29917\crt\src\sanitizer]
I’ve noticed that the Just-in-time-debugger doesn’t seem to fire when an ASAN issue is raised, but when running from within a debugger the debugger catches an exception and stops the process at the point of failure.
Is there a way to get ASAN issues to fire the just-in-time debugger, like other unhandled exceptions do?
Hi Stephen, as of now we don’t offer that support. Can you please create a suggestion ticket here: https://developercommunity.visualstudio.com/report?space=62 ?
There is additional work needed to make this work: we have special VS debugger handling for ASan exceptions, and it may be tricky to get it to play nicely with the JIT popups. If enough folks upvote the ticket we can prioritize this work.
Thanks,
Eric Brumer
You wrote that your test suite contains Qt Creator. Does that mean that you were able to build Qt with address sanitizer enabled? If so, which version of Qt? Could you please share a build script or at least some instructions for that?
Hello Markus,
We are currently building commit da3b14f from the repo at git://code.qt.io/qt-creator/qt-creator. We build most projects with ASan by setting the _CL_ environment variable to /fsanitize=address /Zi /FS /GS-, and the _LINK_ environment variable to /InferASanLibs /incremental:no /DEBUG. Setting these variables will append the ASan command-line switches to all CL and Link invocations without needing to modify build files. ASan-enabled QT Creator can then be built using the same steps as regular non-ASan builds without the need for a special script.
Note that our automated test suite builds QT Creator with ASan, but doesn’t execute the resulting binaries. Most of our tests also execute the produced binaries, but for QT Creator this is not the case currently. This still helps us validate our ASan implementation by confirming that no internal compiler errors occur during the build itself.
I hope this helps, and please let us know of any feedback you may have if you end up using ASan to build and run QT Creator.
Kevin Cadieux
Software Engineer
MSVC Team
Hi There,
We have a legacy product with lot of libs, dlls, and exes. We have migrated these solutions to VS2019.
Does the option /fsanitize=address to be enabled in all the projects.
When I try this option in one of the dll project from General page->Enable Address Sanitizer, I’m getting the linker errors . I tried to including the VCASAN.lib , but still the linker errors are coming. Could you please help to resolve these errros.
Severity Code Description Project File Line Suppression State
Error LNK2019 unresolved external symbol _PssCaptureSnapshot@16 referenced in function “int __cdecl __vcasan::DumpThisThreadExceptionFilter(struct _EXCEPTION_POINTERS *,struct __vcasan::EXCEPTION_SANITIZER_ERROR * *,unsigned short const *,unsigned long &)” (?DumpThisThreadExceptionFilter@__vcasan@@YAHPAU_EXCEPTION_POINTERS@@PAPAUEXCEPTION_SANITIZER_ERROR@1@PBGAAK@Z) wizapi D:\ADO\InTouchNEWUIUX\src\sc-core-Irvine\wizapi\VCASAN.lib(vcasan.obj) 1
Error LNK2019 unresolved external symbol _PssFreeSnapshot@8 referenced in function “int __cdecl __vcasan::DumpThisThreadExceptionFilter(struct _EXCEPTION_POINTERS *,struct __vcasan::EXCEPTION_SANITIZER_ERROR * *,unsigned short const *,unsigned long &)” (?DumpThisThreadExceptionFilter@__vcasan@@YAHPAU_EXCEPTION_POINTERS@@PAPAUEXCEPTION_SANITIZER_ERROR@1@PBGAAK@Z) wizapi D:\ADO\InTouchNEWUIUX\src\sc-core-Irvine\wizapi\VCASAN.lib(vcasan.obj) 1
Error LNK2019 unresolved external symbol _PssQuerySnapshot@16 referenced in function “int __cdecl __vcasan::DumpThisThreadExceptionFilter(struct _EXCEPTION_POINTERS *,struct __vcasan::EXCEPTION_SANITIZER_ERROR * *,unsigned short const *,unsigned long &)” (?DumpThisThreadExceptionFilter@__vcasan@@YAHPAU_EXCEPTION_POINTERS@@PAPAUEXCEPTION_SANITIZER_ERROR@1@PBGAAK@Z) wizapi D:\ADO\InTouchNEWUIUX\src\sc-core-Irvine\wizapi\VCASAN.lib(vcasan.obj) 1
Hello,
thanks for your work.
We have a solution of 230 projects and some of them are linked against a third- party libraries which we have in a binary form only. Is there any way to disable asan exceptions in these libraries which we cannot recompile on our own? (any black list as in CLang?)
When opening a plain Win32 file open dialog in a plain Win32 application and ASAN enabled, we are getting various exceptions from inside windows.storage.dll calling memset() inside ucrtbase.dll. If we do ignore this issue, we are getting some more of various exceptions. If not running under debugger, the application just closes. We use ASAN on other platforms (OSX, Linux), but on Windows we are still out of luck due to this issue. We don’t know whether there are other ASAN issues that are critical, because we cannot get beyond the file dialog issue.
This is the snippet that makes a Win32 application crash in ASAN mode:
OPENFILENAME ofn; // common dialog box structure
TCHAR szFile[260] = { 0 }; // if using TCHAR macros
// Initialize OPENFILENAME
ZeroMemory(&ofn, sizeof(ofn));
ofn.lStructSize = sizeof(ofn);
ofn.hwndOwner = NULL;
ofn.lpstrFile = szFile;
ofn.nMaxFile = sizeof(szFile);
ofn.lpstrFilter = _T(“All\0*.*\0Text\0*.TXT\0”);
ofn.nFilterIndex = 1;
ofn.lpstrFileTitle = NULL;
ofn.nMaxFileTitle = 0;
ofn.lpstrInitialDir = NULL;
ofn.Flags = OFN_PATHMUSTEXIST | OFN_FILEMUSTEXIST;
ofn.FlagsEx = OFN_EX_NOPLACESBAR;
if (GetOpenFileName(&ofn) == TRUE)
{
// use ofn.lpstrFile
}