Microsoft C++ Code Analysis supports SARIF 2.1

Hwi-sung Im

Starting with Visual Studio 16.8, MSVC Code Analysis officially supports SARIF 2.1.0 standard. SARIF is an industry standard for representing static analysis logs and we’ve been one of the earliest collaborators with the SARIF Technical Committee. Behind the scenes, analysis log files in the SARIF format powers Visual Studio IDE to provide a richer experience that was not possible with the legacy XML logs. In this blog post we will present one of those enhanced experiences – code analysis defects with different severity levels in Error List and color-coded squiggles for offending code segments. 

What is SARIF? 

SARIF is an acronym for the Static Analysis Results Interchange Format, which is a standard, JSON-based format for the output of static analysis tools. SARIF provides a rich format intended to meet the needs of a wide range of analysis tools, both sophisticated and simple ones. It also provides an extensibility mechanism to allow tool authors to store custom data that the SARIF format doesn’t directly support. 

You can find more information on SARIF at this introductory page. The latest SARIF standard is available at this page. 

C++ Code Analysis and SARIF 

MSVC Code Analysis has been using a custom XML log format to report defects it finds during code analysis. While this has served customers well for the last decade, we decided to support SARIF for richer capability such as logging list of files analyzed, configuration overrides for analysis, location where defects are suppressed if logging suppressed warnings, etc. 

We have been supporting SARIF in VS for quite some time, even the earliest drafts of the specification. 16.8 is the release that MSVC Code Analysis supports SARIF 2.1 standard. With this update, MSVC Code Analysis provided some enhanced experience that many readers may already know, including but not limited to: 

• Background code analysis 
• Green squiggles for code analysis defects 
• Fix-it for automatic recommendations to fix defects 

To add more improvements to code analysis, we have updated MSVC Code Analysis to support the latest SARIF 2.1.0 standard. At the same time, it has been updated to provide options to record more information to the SARIF log file: 

• List of files analyzed  
• Configurations used for code analysis, including command-line options 
• Rule actions for each of the active rules for which defects are reported 

While we were updating MSVC Code Analysis to add more information to SARIF log files, we also added new capabilities to it: 

• Ability to log compiler warnings in addition to code analysis warnings 

• Ability to log suppressed warnings with location of suppression 

Rule Actions in Visual Studio 

As we update MSVC Code Analysis with support of the latest SARIF standard and additional information on analysis and defects, we also updated Visual Studio to enhance the code analysis experience.  

In the following sections, let’s look at an enhanced code analysis experience in Visual Studio IDE. 

Ruleset and Rule Actions 

MSVC Code Analysis can be used with one or more ruleset files that can enable or disable selected rules. Rules that are enabled in the ruleset are checked and reported by the analysis tools. Rules that are disabled in a ruleset will be ignored by MSVC Code Analysis and will not be checked or reported. More information on rulesets can be found here. 

Each of the rules in the ruleset file can be assigned with one of the following “Rule Action” values: 

Until we added support for SARIF, MSVC Code Analysis could only map these rule actions simply to “enabled” or “disabled” states, and either reported or ignored until it was added with the support for SARIF. Visual Studio build used ruleset files directly and could differentiate rules with “Error” Rule Action from the reset of the enabled rules, which were considered as “Warning” regardless of their Rule Action values. Visual Studio IDE did not honor the Rule Action values, and treated everything as “Warning”. 

With the support of SARIF, MSVC Code Analysis now maps these rule actions to the SARIF’s level property in the log file as follows: 

A rule with Rule Action value of “default” will be assigned with one of the other Rule Action values before mapped to the SARIF’s level. 

Rule Actions in Action 

Let’s look at how Visual Studio now uses the Rule Action values in the ruleset to improve the code analysis experience. As an example, let’s consider a custom ruleset with the following content: 

<?xml version="1.0" encoding="utf-8"?> 
<RuleSet Name="Copy of Microsoft Native Recommended Rules" Description="My custom rules" ToolsVersion="16.0"> 
  <Rules AnalyzerId="Microsoft.Analyzers.NativeCodeAnalysis" RuleNamespace="Microsoft.Rules.Native"> 
    <Rule Id="C6001" Action="Warning" /> 
    <Rule Id="C6011" Action="Error" /> 
    <Rule Id="C6201" Action="None" /> 
    <Rule Id="C6385" Action="Info" /> 
    <Rule Id="C6386" Action="Info" /> 
  </Rules> 
</RuleSet>

And a *.cpp file with following test code: 

#pragma warning(disable:4700)
  
void foo()
{ 
    int arr[4]{}; 
    int v = arr[4]; // C6201, C6385 
    arr[4] = 2;     // C6201, C6386 

    int x; 
    int* py = nullptr; 
    int y = *py;    // C6011 
    y += x;         // C6001 
}

When configured to use the above custom ruleset for analysis, analyzing the test code in Visual Studio produces results as shown in below screenshot: 

Please note how the defects are listed in the Error List window, and how their corresponding code segments are highlighted in the text editor window: 

• In the Error List window, C6011 is listed as an Error, C6001 as a Warning,  and C6385 and C6386 as Messages. 

• In the text editor window, code segment for C6011 is highlighted with red squiggle, C6001 with green squiggle, and C6385 and C6386 with gray squiggles (dots) 

As you can see in the above example, rule actions in the ruleset maps to Visual Studio build output, Error List, and text editor display through the level property of SARIF as follows: 

Please note that rules with Rule Action value of “Info” is also reported as “Warning” in the Visual Studio build output window. This has been the behavior before the adoption of SARIF log format. Following our guiding principle of keeping minimal friction for our customers during Visual Studio upgrades, we kept this behavior as is to not change the build behavior of existing projects. Please share your feedback on this for future improvements – whether it should be updated to honor the Rule Action value of “Info” and reported as “Note” or something like that. 

Send Us Feedback 

Please download the latest Visual Studio 2019 and give it a try! Any feedback is welcome. We can be reached via the comments below, Developer Community, email (visualcpp@microsoft.com), and Twitter (@VisualC). 

Hwi-sung Im Senior software engineer at Microsoft, C++ Static Analysis Team

Follow