But Australian Telecom loves it!

There are two popular ways of measuring how widespread a problem is: You can look at the number of times the problem occurs, and you can look at the number of users it happens to.

Back in the late 1990’s, I remember a presentation from a product team on things they learned from their telemetry. One of the most important things gathered from telemetry was deciding which bugs were most important to fix. (Related reading.)

The presenter displayed a chart of the most frequent failures, and for the purpose of discussion focused on one of them. The failures were plotted on a distribution curve, where the height of the curve showed the number of times an individual computer encountered it. And although the total number of failures was high, the distribution curve revealed that pretty much all of the failures were coming from a single computer. The problem wasn’t so much that the feature wasn’t working for a lot of people; rather, the feature was working great for everybody, except this one poor guy where the feature never worked at all.

Why wasn’t the feature working for that guy? Who knows. Maybe his hard drive is corrupted. Maybe he has a computer virus. But whatever is wrong, it’s affecting just one person. (And since telemetry is gathered anonymously, we have no way of contacting him to find out. This was before we had a way of asking people to leave their phone number so we could call them back with further questions.)

In a similar way, one user complaining 10,000 times is worth far less than 10,000 users complaining once each. (And psychologically, one user complaining 10,000 times is actually worth less than one user complaining once, because one user who complains 10,000 times starts to sound like a crackpot.) And a corporation with 10,000 employees can say “This affects all of my 10,000 users,” and they effectively cast 10,000 votes.

But you have to be careful you don’t give too much weight to the corporations who can cast 10,000 votes at once.

Windows 95 came with an email client known as Windows Messaging. That was the thing that ran if you double-clicked the Inbox icon on your desktop. This component was developed by the Exchange team, and sometimes I would send them feedback saying “When I use your program to read my MSN mail, XYZ happens, but that doesn’t really make sense for MSN.” And the Exchange folks dismissed the feedback by saying, “But Australian Telecom loves it!”

Apparently, Australian Telecom (now known as Telestra) was a large install base for Windows Messaging, with tens of thousands of employees. That meant that when they had some feedback, it carried the weight of 10,000 users.

I didn’t have the heart to tell them, “That’s so cute. You know who your largest customer will be once Windows 95 ships? MSN, with 250,000 users.”