Testing Compiler Diagnostics…and a Cool IDE Tip!

Visual CPP Team

Hello everyone, this is Jamie Eckman.  The last time I posted I was an SDET (tester) on the Libraries team.  I’ve since moved to the Compiler team, still as an SDET.  One thing we see a lot of on both teams is compiler diagnostics: warnings and errors.

One of our Libraries developers, Stephan T. Lavavej, wrote this last year on one of our internal discussion lists: Figuring out the mapping from “what the compiler is complaining about” to “what’s actually wrong with my code, if anything” is like 75% of learning how to program. Depending on how you learned to program, 75% may or may not be an overstatement, but the point is clear.  Everyone writes bad code sooner or later, and in the case of new programmers it’s sooner (and often).  For this reason it’s important that the errors the compiler emits are correct and clear enough to diagnose the problem.  Clarity is something we’re making a significant investment into improving over the next few releases of Visual Studio.  For now I’ll mostly stick to talking about correctness.

When I was still on the Libraries team the only direct contact with diagnostics I had was some maintenance of the ATL Server attributes error tests.  But for the most part I looked at compiler errors from the end-user perspective:  This test I wrote doesn’t compile, why?  Now that I’m on the Compiler team, which owns the bulk of the error text you’re likely to see while writing C or C++ code, I’ve had a great opportunity to look at up close how our diagnostics work and how we test them.

The compiler has on the order of 1,000 errors and warnings it can emit.  We test these like you might expect, each warning or error has one or more test cases associated with it.  The test cases verify that the diagnostic that’s expected is actually emitted and is emitted on the correct line.  Of course, the compiler is a hugely complex mechanism and problems are bound to slip through the cracks.

A colleague on our UE team (User Education, responsible for the content on MSDN, among other things) recently came to me with a small list of code snippets from the C/C++ Build Errors section of MSDN that weren’t producing the promised errors.  These snippets are pretty useful when you’re trying to figure out what a particular error or warning means.  Oftentimes the simplified code in the snippet can help you spot the problem in your own code.

So, Tim from the UE team asked me to look at these snippets that weren’t working and figure out what was wrong.  A compiler bug, an expected change we forgot to tell UE about, or something else?  As an example, here’s the first one I looked at:

// C2062_b.cpp

// compile with: /c

class DataSent {};


struct MyDataController {

   MyDataController(int age, DataSet* j) {}   // C2062

   // try the following line instead

   // MyDataController(int age, DataSent* j) {}


This is a simple error, although perhaps not easy to spot.  The class name “DataSent” has been mistyped “DataSet”.  The expected error is C2062, “type ‘DataSet’ unexpected”.  The error that actually gets emitted (with a recent Orcas compiler) is C2061, “syntax error : identifier ‘DataSet’”.  Here we see that correctness and clarity often have an interesting relationship when it comes to error messages.  Although we didn’t get the error we expected, the error we did get nonetheless points us directly at the problem.  I might be able to make the case that this is a compiler bug, but because the new error is functionally just as good as the old error, it’s unlikely to be fixed at this stage in the Orcas release cycle.

Now, here’s a tip that has abso lutely nothing to do with the rest of this post.  Some of you may already know this, but I haven’t seen it posted on vcblog yet.  I recently started working in a new codebase with a pretty strict set of coding conventions.  One of the conventions is that nothing should go past column 79.  It’s pretty annoying to watch the column display in the status bar every time you get close to the end of a line, but fortunately there’s a nifty hidden feature of Visual Studio that can help.  Column guides.  This involves editing the registry, so the standard disclaimer applies: Modify the registry at your own risk!

Find this key in your registry:

HKEY_CURRENT_USERSoftwareMicrosoftVisualStudio9.0Text Editor

For Visual Studio 2005 you’ll want to use “8.0” instead of “9.0”.  Create a new String Value in the Text Editor key and call it “Guides”.  Modify the data for Guides and add something like this:

RGB(0, 128, 0) 79

This will put a green line at column 79.  You can have multiple guides:

RGB(0, 128, 0) 79, RGB(128, 0, 0) 49

Make sure you restart Visual Studio after you add these values for the changes to take effect.


– Jamie


Discussion is closed.

Feedback usabilla icon