In real life, nobody upgrades their server
When you install Windows on a machine that already has a copy of Windows, you get the option of performing a “clean” install (installing the operating system from scratch, ignoring the existing one) or an “upgrade” install (upgrading the existing operating system to the new one).
And in real life, it turns out, surveys show that nobody upgrades their servers.
Sure, IT administrators may get new hardware, they may upgrade the software, they may even install a new version of the operating system, but our research says that, to within experimental error, nobody does an “upgrade install”. They always do a “clean install” and reinstall any applications that they need. Since this is a server, there aren’t many applications that need to be installed, so reinstalling all the applications doesn’t take quite so long.
What is the lesson here?
You have to dogfood intelligently. If you’re self-hosted on a server build and you install the latest build via an “upgrade install”, you’re not really helping to test setup, since no real customer does what you’re doing. This type of disconnect has historically been a problem in the operating systems group, where people typically run through the interactive setup (or if they’re really clever, hand it an unattend script to answer the questions automatically), even though research shows that less than ten percent of all Windows systems are set up that way. The vast majority of the systems are installed via a process known as Sysprep. But inside Microsoft, very few people use this Sysprep install mechanism. Result: More attention is paid to improving the quality of manual setup at the expense of Sysprep because more internal bugs are filed against manual setup than against Sysprep. Even though Sysprep method is more commonly used in real life.
In other words, we failed to dogfood setup.
Fortunately, with Windows Vista, the setup team recognized this disconnect between what gets used at Microsoft and what gets used in the real world and attacked it head-on: The two setup models have been merged into one. (And, if you count “clean install” and “upgrade install” as two different models, which they sort of are, then in fact three setup models have merged to one.) The three types of setup (mass-produced, manual clean install, and manual upgrade install) are now just minor variations on a single engine.
The dogfood is back.