We Need to Talk (Matt Gertz)

Anthony D. Green [MSFT]

By now, many of you will have downloaded the recent beta that we posted to the net, and (hopefully) are getting excited about the features that we’ve been developing.  Recently, Beth Massi interviewed me about some of the work that I’ve been doing as a part of that effort.  I always enjoy interviews, and Beth is great at pulling out some of the more interesting aspects of a person’s work, but I’m always somehow left with the feeling afterwards that there’s something more that I wanted to say — some different issue to which I forgot to draw folks’ attention.  You should therefore think of this blog entry as a follow-up to that interview, where I get to spend some time discussing the people-aspects of the job.

“People, process, product.” 

(variously attributed in current management theory)

In the course of the aforementioned interview, I spent a lot of time discussing process, and of course this blog site spends a lot of time focused on product.  I haven’t talked much about the “people” aspect of it – and yet, that’s a really important part of the triad.  The product needs to address what people (our customers) need and what other people (our employees) can deliver.  The process, in turn, needs to be defined in such a way as to deliver the product to the people.  Process for process’ sake will inevitably result in the wrong product, which in turn will greatly annoy both the people developing the product as well as the people for whom the product is intended.

That’s not to imply that we don’t spend a lot of time on the “people” issue – in fact, it’s the needs of our customers that drive us to create the best software we can.  I remember the first time I picked up a Visual Basic magazine and thumbed through it.  As I read it, I noted that many of the articles and “letters to the editor” involved features that I had written or that I somehow was responsible for.  And as I continued to read, I was suddenly struck by an almost frightening realization:  the effectiveness of my work, and of my team’s work, had a direct impact on the daily lives of our customers. 

Not just “customers.”  Millions of customers.  Yep, that’s a big number.

This was something that I suppose I’d always understood in an abstract sense, but reading this magazine really brought it home for me.  Previously, I’d joked about how I was getting paid by Microsoft for doing what was essentially a hobby for me – writing software for fun and profit, what a blast!  Suddenly, I realized that there was something bigger going on – keeping electric plants working efficiently, allowing the initiatives of various governments to run smoothly, enabling small businesses to keep on an even keel – it’s just an incredibly diverse ecosystem that uses our software.  I got a lot more serious about my attitude towards coding after that personal epiphany, believe me.  It’s a sobering thought as to how the work that any of us does impacts others, and it can be highly embarrassing when we get something wrong.  This is why, over the years, we’ve worked closely with not only MVPs but with a broad spectrum of our customer base to determine where we need to focus our attention.

Most of my “people” work these days, however, is focused within Microsoft – how we as engineers interact with each other in such a way that we can produce a better product.  Currently, within our division, we have over 30 product groups that somehow need to work together as seamlessly as possible for us to stay on track.  Visual Basic is a big part of that, but there are many other parts as well, and they all connect together in interesting ways to cover the broad spectrum of scenarios that we support.

 “What we’ve got here… is a failure to communicate.”

(Luke, in Cool Hand Luke, mocking the prison captain’s earlier words)

Over a dozen years ago, we started the first efforts to bring together the various pieces that would one day make up Visual Studio.  The first big step that we took was to start focusing attention on the “Vegas” shell (our internal codename for the shell that we still use in VS today) – all new products would leverage this shell for Visual Studio 6.0, which other established products (notably Visual C++ and Visual Basic) would use their old shells but would have some of their functionality supported by the new shell (which would, for example, wrapping their build functionality via special project types).  This pulling together of different groups meant that the “build manager” position for each team suddenly went up in importance.  Previously, this person simply needed to make sure that dependencies were in place and that daily builds got built.  Now, this person instead had to work directly with peer teams to pass code around and integrate it.  The problem was, each of these teams had different cultures and different processes doing things, and they’d never been pressed to evolve the skills required for this new sort of relationship.

At the time, in addition to my coding duties, I’d acquired the build manager position for the VS Enterprise Edition product unit (comprising source code control and deployment teams), and thus the responsibility for coordinating with my peers on these other product units.  I can best describe those days as “dark.”  This was a challenging time for us, and most of the challenges arose because of a lack of communication between the teams.  Inevitably, whenever I would integrate some other teams’ code into our codebase, their code would have massive conflicts with our code, and I would spend 2-3 days getting things working before being able to return to my “normal” programming duties.  There were days where I and others worked very, very late to get things working again.

The other teams didn’t mean to break my team’s code.  Our team didn’t mean to break their code.  The problem was simply that it never occurred to anyone that their change would have a ripple effect with other teams in this shared environment, nor that they needed to communicate such changes.  Now, you could claim that there was a process problem involved – and you’d be right.  Our processes changed substantially as a result of this, believe me.  But the fundamental problem was a cultural (i.e. people) problem – each team was still acting as if they were on their own island, and every divisional problem took a backseat to their drive to provide their own core functionality.  There was no common language to describe the importance of one team’s scenario with respect to another’s.

Now, since then, we’ve made huge strides in inter-team communication.  Leaders from across the division, in all disciplines, regularly meet to discuss changes coming down the line.  The build manager position has evolved into the Build Facilitator Developer position (BFD), and they (sponsored by my team) also regularly meet to discuss policies and practices.  The division as a whole devises specific goals for a given product that teams need to incorporate into their feature planning.  As a result of improved processes and better communication, our downtime for build issues is the merest fraction of what it was a dozen years ago.  We’ve continued to evolve these practices which, along with new processes, means that even the difference between now and a few years ago is stunningly better with respect to build efficiency.

“I feel that if a person can’t communicate, the very least he can do is to shut up.”

(Tom Lehrer, That Was the Year That Was)

So, in the previous section, the communication I discussed was mostly tactical in nature – i.e. here’s what’s coming, and here’s what we’re going to do to avoid problems.   Another aspect of communication, though, is letting people where we are right now.  It’s a critical point, because the best way to avoid mistakes, or to recognize the things that worked and which should be made policy, is to fully understand how our past work led us to this point.

Now, my first attempts along these lines, back when I was the Visual Basic box lead, were not as successful as I would have liked.  In my eagerness to get all of the information out there, I evolved what started out as a small weekly bug report into this huge, massive thing, comprising about 6 pages of data which covered every single aspect of the product’s state.  If you wanted to know *anything* about the status of Visual Basic, you could find it in that weekly mail.

The problem was, no one read the whole thing.  It was simply too much information — you couldn’t look at the report and know what data was important.  It wasn’t communication at all – it was death by drowning (in data).  Added to this was the problem that I (still) tend to be a very verbose person, using two words where one would suffice.  It actually *hurt* communication because I still had people asking me for data hidden away in my mail – I was wasting my time and theirs.

Eventually, something clicked.  I found out that people were at least reading the first paragraph, where I’d provide a brief recap of the previous week and thank the people who’d specifically helped out with the problems therein.  They read that part because it was *personal* — it related to them, not in an esoteric “here’s how many bugs you have” context, but instead in a familiar “I did this” sort of way.  So, a few years later when I assumed my current job (which eventually included, among other duties, providing daily status to the divisional staff), I decided to re-think how I did my status reports.  The status I or others on my team send out is devised as follows:

          What happened yesterday, why it happened, and how does it affect you

          What’s happening today, and what the status is

          What the most important resources are currently (i.e., builds and tools), and how you can access them

          What are the most important issues to address today

That’s it – the whole thing takes less than a printed page of text.  It is a person-oriented mail, which gives kudos to folks that did something particularly well, and points out (in a clear but respectful way) areas where things could have gone better.  Additionally, I sometimes add a “Question of the Day” which answers (in 1-2 small paragraphs) a process-relevant question which one or more people have asked me recently.  This gets the focus on the issues that really matter to people in the division, and not just the ones that I think matter, so that everyone is made aware of them.  This format of status mail has been received much more favorably than the previous one.

Let’s talk

Hopefully, some of what I’ve said above is relevant to the work that you do.  I, in turn, would be fascinated to learn more about how you, our customer, tackle the issue of communication both inside and outside your own business.  I’m certain that we’ve got a lot more to learn from each other beyond coding – the process of coding is really important, but without understanding how to communicate with the people involved, the code you and I create might just be the best thing that no one’s ever heard of.

‘Til next time,



Leave a comment

Feedback usabilla icon