Filling in some gaps in the story of Space Cadet Pinball on 64-bit Windows
Space Cadet Pinball has a special place in the hearts of many Windows enthusiasts. A customer used their support contract to ask how to change among the three levels of play in Space Cadet Pinball. My proudest achievement of Windows XP was fixing the game so it didn’t consume 100% CPU. People keep asking if it can be brought back.
One point of contention is over my claim that I removed Pinball from Windows because I couldn’t get the 64-bit version to work. Retrocomputing enthusiast NCommander even undertook a Zapruder-level analysis of all of the 64-bit versions of Windows he could find to prove or disprove my story.
I was amazed at the level of thoroughness (and the fortitude it required to get those Itanium systems up and running, much less debug them), but there’s one version of 64-bit Windows that NCommander didn’t try out, and that’s the one that’s relevant to the story.
When the 64-bit Windows project started, there was no Itanium hardware yet. The only way you could run any Itanium code was to run it in a simulator. Booting Windows on an Itanium simulator took forever. Clearly not the development environment you wanted when porting millions of lines of code.
On the other hand, the Windows team did have access to a lot of copies of another 64-bit processor: The Alpha AXP.
In 1999, Compaq announced that it would no longer support Windows on the Alpha AXP, which left a lot of Windows team members with cumbersome paperweights on their desks.
Let’s see what we’ve got here.
- A lot of Alpha AXP machines sitting around with nothing to do.
- A bunch of developers who have experience with the Alpha AXP.
- A 32-bit version of Windows that runs on the Alpha AXP.
- No Itanium hardware on the immediate horizon.
- A ticking clock.
The Alpha AXP is internally a 64-bit processor. It’s just that 32-bit Windows used only the 32-bit subset.
Solution: Port the Alpha AXP version of 32-bit Windows to 64-bit Windows.
Now, 64-bit Windows on the Alpha AXP would never ship. But the Alpha AXP did have the advantage of existing in physical form, so the system could finish booting before the heat death of the universe. The assumption is that most of the effort in porting Windows to the Itanium is in the 32-bit to 64-bit transition, and not in dealing with quirks of the specific 64-bit processor you are porting to.
The assumption was somewhat validated by experience: The 32-bit Windows code base had been ported to many 32-bit processors, with relatively few architecture-specific issues. Once you got a 64-bit version of Windows working for the Alpha AXP, it should be a comparatively small amount of additional work to port it to Itanium. The hard part was going from 32-bit to 64-bit.
The team set to work, and we had 64-bit Windows running on physical Alpha AXP hardware long before we had any physical Itanium hardware. I could test my 64-bit port on a physical Alpha AXP system to validate that it was successful. And that’s the system that had the broken collision detector.
NCommander did find a collision detection bug on the Itanium, although that bug was nowhere as severe as the one that existed on the Alpha AXP. My guess is that it had to do with the default rounding mode established by the C runtime library.
My theory as to what happened is that some time after I removed Pinball from the product, the C runtime team realized that they had a compatibility bug in the way they set the default rounding mode, and they fixed it. Or maybe there was a compiler bug, and the compiler team fixed that. Whatever the problem was, somebody fixed it, and then they went back and re-tested Pinball with this fix, and everything worked great, so they put Pinball back.
I’m just guessing about what happened afterward because nobody informed me that they had gotten Pinball working and added it back. I just assumed that it was gone forever.
While I understand the economics of why the Alpha situation happened (and would always have happened for the same reason MIPs and PPC went away) there will always be a part of me that wonders “what could have been” if the 64bit version of NT had hit the market instead of the 32bit version and the benchmarks really did justify the expense.
That said MIPS, PPC, Alpha AXP, and even to an extent ARM/ARM64 all face the same issue: cost (and compatibility). Because there is no off the shelf solution for any of these things with a competitive second source they will always struggle to gain market share because the costs will always be higher for the performance than the x86-64 world is. Intel probably doesn’t realize just how lucky they are that IBM insisted on a second source and they licensed the architecture. Without that who knows what we’d be running? x86 doesn’t have to be the best, it just has to be the cheapest for the performance. That was what ultimately killed MIPS, PPC, and Alpha despite all of them having some advantages.
ARM64 has multiple sources – which is part of why I think it might be a runner. In addition to ARM’s designs, Apple and others have invested heavily. At the same time, the low power background of ARM is really proving to be an advantage for performance on these new process nodes. ARM is also the dominant offering in a mass market sector of mobile devices while PPC never really had that given it was used either in IBM etc. servers or in the much smaller Apple market.
Not in the same way x86 did at the start. I don’t think x86 could get going today with the way hardware works as easily. The fact that AMD/Intel/Cyrix/IBM CPUs were all relatively interchangeable until the death of Socket 7 was a huge boon to market penetration. Whereas far as I know there is no socketed ARM64 solution much less one with a compatible UEFI that I can just install Windows on. Every single ARM64 board is different and has to be qualified because they all do different things. If you notice ARM64 is succeeding only in closed ecosystems where the entire experience is managed it’s not really making inroads into the larger market.
Apple M1 is the only performance competitive ARM64 CPU on the market and its single vendor single source closed system. It’s really no different than the Commodore 64 in that regard. The only thing it has going for it over the C64 is that MacOS abstracts all the hardware so it can remain ‘compatible’ when Apple changes things going forward.
But as far as I know there is no readily available multi sourced option anymore. Technically even x86 doesn’t have multi-source beyond compatible ecosystems anymore. I can’t run an AMD Socket 1200 CPU or a Intel Socket AM4. That means that the costs of those ecosystems is far greater than it should be because of the lack of competition. I remember when Nvidia chipsets were the best overall, intel was just OK, and VIA was the budget option. Sadly those days are long past.
The economics of a lot of things are different now. AWS has a custom ARM chip available now. There are ARM servers available from various companies. In many markets that didn’t really exist in the 1980s like web applications etc. recompiling isn’t really that much of a burden if that’s even necessary given that most will be targeting a runtime like JVM, CLR or Node, which only needs to be ported once. Even computational applications written in native code can take advantage of reduced costs from cloud vendors or even on prem due to reduced cooling requirements etc.
IA-64 had no second source: that was part of the reason for its existence. That meant AMD had to create a new architecture, did a decent job, and that’s where x86-64 came from. Intel could use it because of their cross-licensing agreement on x86, and they did. According to an AMD senior engineer who visited my employer, Intel originally wanted to make their version of x86-64 deliberately and thoroughly incompatible with AMD’s, hoping to crush AMD with marketing. Dave Cutler at Microsoft killed that idea, by telling Intel that Windows would not support such an architecture. It was a silly idea anyway: network effects create huge benefits from software compatibility.
Compaq killed Alpha because they believed Intel and HP’s marketing about IA-64, and wanted to stop investing in Alpha.
Yeah Intel was quite explicit in their monopolistic intent with IA-64 to drive AMD and any other licensees out of the market. But much like IBM and MicroChannel it worked just as well because the market just ignored it. There is a long discussion that can be had comparing AMD64 design decisions to ARM64. But I think that’s for Raymond to pick up at his leisure at another date.
As for Compaq stopping their investment in Alpha? Yes it was Intel/HP’s “Marketing” totally not the fact that Intel threatened Compaq’s allocations at all… totally. Compaq was given quite a lot of financial incentive not to buy AMD or to continue making Alpha at the time, something Intel was eventually taken to task for and not just for Compaq. But by the time that happened the damage was long since done and irrecoverable.
Dave Plummer’s popular Youtube channel (Dave’s Garage) has a video “Windows 11 Pinball: by the original Microsoft programmer of the XP Game” where he give a lot of the history and attempts to get it running on Windows 11 (watch the video to see if he succeeds or not).
On a related note – when is Raymond going to do a livestream with Dave? It could be an AMA.
I believe Windows NT 4.0 for Alpha had Pinball. Do you know why that version did not have the same problem?
Unless I’m misunderstanding, I think this is the answer:
“Now, 64-bit Windows on the Alpha AXP would never ship.”
They created the 64-bit port of Windows to Alpha AXP to just have something to test on, but it was never publicly shipped.
The 32-bit version(which I’m assuming is what was publicly released, based on what’s said in this post) never had that problem in the first place, and Raymond’s first post on the issue specifically made it clear that it was when it was being ported to 64-bit that the issue started occurring.
If you’re asking why the 32-bit didn’t have that problem when the 64-bit did, well, I don’t know enough about the processor to know, and Raymond even highlights in this post that it was likely platform decisions or issues(whether with the runtime library, or the compiler), but the exact cause is, presumably, lost to time.
This reminds me of when Jet 4.0 updates for Windows 7 were compiled with /arch:sse2.
I wonder if there is any chance DEC/Compaq/HP can pull off what Apple is doing with the M1 today back then.
So, I didn’t expect to ever see a follow-up to my video from Raymond himself, and honestly, I apologize for not reaching out to you directly. I want to write a quick note before this gets buried, so I apologize if this is more stream of consciousness. There were some points I actually left out of my research because the video was getting extensively long, but I’ll try to summarize my notes here. I have probably at least 10 or so copies of Pinball sitting in my projects folder which is a testament to the madness I undertook.
In the process of reaching through Pinball versions, I actually ended up going back to NT4, and validating that all four ports had working Pinball, and seeing FPU before on DEC Alpha was 32-bit precision vs. the 53-bit precision of x86 (and played correctly. I actually had suspected the collision bug you talked about had been introduced in the AXP64 port, and then carried forward to Itanium, because the Windows 2000 DDK seems to suggest that _M_AXP64 used a 64-bit precision mode like Itanium. I left AXP64 out of the video because I suspect talking about Itanium was going to cause a lot of people’s eyes to glaze over as is, and then I need to explain that a never shipped port of Windows exists and … yeah …
I actually did spend more effort than I care to admit to fix the CtD bug on the XP RTM IA64 pinball.exe, but between WinDBG randomly crashing, and Itanium assembling being madness inducing, I didn’t get too far with it. It doesn’t help that I don’t have IDA, and Ghidra doesn’t understand IA64 either. From what I could tell, I think its choking on a data size mismatch with trying to load the collision elements from pinball.dat, since I see a lot of magic numbers which seem to assume 32-bit behavior which were cleaned up for XP 2003, but I won’t pretend to be an expert in reverse engineering.
For quite a long time, I had actually suspected that what had happened was the bug got fixed on the svr2003 branch sometime after Longhorn had forked off, and had gotten lost in the post-reset builds, which is why XP x64 and Server 2003 had a working Pinball game. My notes page goes into rather insane detail tracking how version numbers and builds were, to figure out what versions did and didn’t work. Looking back, I question my sanity on how deeply I dove. I actually did try multiple versions of the IA64 compiler to see if I could find a regression, but failed.
The fix seems to have introduced sometime before Server 2003 was branched off, because that’s when the installer bug was introduced which was when Pinball was re-added to AMD64 builds, and that mistake was present in build 5001, which was the first post-reset build going back to the svr2003 base.
I may actually do a follow-up at some point, incorporating this, and larger details on what happened, as a few other people have pointed me at things I didn’t know. If nothing, I would love to put a more definitive story up on what happened, and perhaps cover those elements more in-depth (especially since I’ve since obtained DEC Alpha hardware).
A little bonus treat if you go to the comments section of the linked NCommander’s video, two of the original authors of Space Cadet Pinball show up and talk about what/how it was given to Microsoft.
I noticed that, at least for the past few weeks, about half of the DevBlogs are your blogs. If you remembered, I notified you a few months ago that one of your blog has nothing to do with development. You said it was some kind of routing error that make the blog appear in DevBlogs. I think the same thing might have happened. My guess is that most of your blogs meant to appear in a Tab call “The Old New Things”, Not “DevBlog”. If not, then please forgive my comment
I have to manually flag posts as “do not show on the DevBlogs main page”. Sometimes I forget. Sorry.
“Sadly, the license agreement does not permit releasing the game as an independent entity. The agreement was for including Space Cadet Pinball in Windows 95, with options to include it in the Microsoft Plus! pack for Windows 95, as well as extending the distribution rights to successor products of Windows and the Plus! pack.Your 2018 post about Pinball”
The solution is obvious, release the Plus! pack on the Windows Store including a new, 64 bit version of Pinball.
if you actually do this, please credit me 🙂
Wouldn’t even have to be the old Plus! pack if it’s allowed in successors.