Hello from the trenches of Orcas
Hi! My name is Martyn Lovell, and I’m the development lead on the Visual C++ Libraries team. You might have seen me blog
before, (though I never seem to manage to get regular enough). You might also have caught me recently on video on Channel 9. I wanted to give everyone a quick update on the kinds of things my team is looking at right now [you should see blog entries from many of them in the coming weeks]. Remember, I’m discussing things here that haven’t yet met the quality we need for shipping, so things could change or disappear before we ship.
We’re in the middle of the first development milestone for Orcas. If you’re an avid blog-reader, you’ve probably already heard that we’re running our development process a little differently this time, in an attempt to be more agile. We’re putting each piece of functionality into its own ‘feature branch’, where it has to live until it is at a standard that is free of known bugs and much closer to ready to ship. A small set of developers/testers and others focus on the feature until it’s done (we call them a ‘feature crew’) We hope this will keep our shipping branches at a higher quality, and also ensure that things don’t make it into the product until we know they are in good enough shape to not delay our release. Of course, building a product this large is complex and diverse, so not every feature can fit into this model, but it’s the general plan we’re using. So far it’s working out well, but of course the proof of a plan like this will only really come later in the product cycle.
Right now, my team is working on three feature crews, and a fourth is about to spin up. I’ll start by talking a bit about each of these.
MSBuild: One of the things I love about the libraries team is the fact that we ship almost all our sources. I’m generally proud of the quality of our source, and I think it’s good discipline for us to know that everything we do will be seen by our customers. [Plus, of course, it adds value to the essay-length comments I sometimes seem to write – like the one on ATLASSUME in atldef.h
J]. As a more practical matter, shipping source makes it so much easier to debug and understand our libraries. I don’t know that I could ever have learnt as much about MFC by reading docs and books as I did my first couple of years of writing MFC applications and debugging through the library sources. Our libraries build with a very complex and involved set of NMAKE-based build scripts. Use of NMAKE was very practical, since almost everyone understands it, and also was good dogfooding, since the team that owns NMAKE is also part of the VC family. You’d be amazed how complex our build is. The CRT build is especially gnarly.
For some years we’ve been watching with excitement as the MSBuild team built a new build system from the ground up. Several people who have worked for me in the past were involved in the MSBuild effort, and I think they’ve done a great job re-imagining this part of our product from scratch and producing a next generation solution to the problem. We took the decision a few months ago to migrate all our VC builds to use MSBuild, and we’ve had a feature crew working on it for a couple of months now; they’re just getting close to complete. I’m really excited to see the new build coming out. One great result of this has been a total spring-clean of our old builds. Though they are still some of the most complex builds in the division, they’re more comprehensible and better understood than they’ve been in years.
Development process: One of the things we need to do at the start of almost every product cycle is rename our library DLLs. You might wonder why we do this. It’d clearly be simpler for you if we kept our DLL names, and their binary contract, the same across multiple versions. For some parts of our libraries, it’s actually possible to imagine doing that and we seriously consider it. The CRT doesn’t change substantially every version, and because it is a pure-C API, when the CRT does change it can often be done in a compatible way. In our last version we made some non-compatible changes [for example, we widened time_t to 64 bits]. Other parts of our libraries have a greater need to change. Given the way C++ classes version, it’s not really possible to make significant changes to our STL or MFC without re-versioning the DLLs that contain those classes.
There’s also a testing problem with keeping the same DLL name. We’d have to make sure that no bug fix we made, or extension we added, was going to cause a subtle problem for existing applications. And we’d have to make sure that our new versions were tested in all the same platforms and scenarios as the old versions. This is called the platform type compatibility bar (I found a good article on platform versus library types
here – the context is .NET, but the concepts are similar). We attempted to hit this compatibility bar for a couple of VC versions in the mid-nineties, with very limited success. Each new major version that shipped under the same DLL names had significant issues as we accidentally caused problems for existing users of the DLL.
Our experience has taught us that if we are making significant fixes or changes to our library components, we should re-version them. We work hard to remain source compatible, but do not need to worry about code compiled with the previous version of the product running on the current version. Our development process is just finishing the process of renaming the DLLs; we already have private copies of our newly minted, shiny msvcr90.dll and mfc90.dll to play with.
This team is also working on a fascinating internal process change that’ll significantly increase our dogfooding and reduce the complexity of our work. But I don’t have space to go into that here.
STL/CLR: One of my biggest disappointments from Visual Studio 8 is that we didn’t have the resources to complete and ship STL/CLR, our managed, verifiable implementation of the C++ Standard Template Library. Those of you on the beta for that product will have seen an early version of the technology in Beta 2. We worked with
Dinkumware on this project for most of that product cycle, and they did some great work. But in the end we simply didn’t have enough people and time to keep working on it with them while finishing the other features we were building. Retrospectively, it was very ambitious of us to attempt to build this library at the same time as the C++/CLI dialect was still evolving.
We and Dinkumware started work again soon after Whidbey shipped, and now have a version that meets a lot more of our original goals. In particular, since last October we’ve done some fascinating optimization work to understand what really drives the performance of a template-based, generic-based library like this in managed code. We’re not finished yet on performance, but in informal testing we’ve seen some exciting results, including a bunch of cases where the magic of templates combined with the C++ optimizer are able to outperform the .NET Base Class Library even in verifiable code. When we envisioned C++/CLI several years ago, these are the kinds of scenarios that we dreamed of, and it’s exciting to see them coming to fruition.
We’re about to spin up a feature crew to integrate the current version of STL/CLR into Orcas. This will just be a snapshot of our current state, but it should allow us to get feedback from some of you in a future Customer Technology Preview. Lots of things will still be incomplete in that release (because we still have future milestones working with Dinkumware to complete the remaining parts of the library), but it should be enough to enable you to get a flavour of the implementation, and try it out and send us feedback. We’d love to hear your experiences.
Servicing: Another group that’s always running is our servicing team. Its job is to deal with incoming bugs in existing features, create QFEs and service packs, and help internal and external customers who contact us via various routes. We rotate membership of this group through our whole team, which gives everyone a chance to work on the broad spectrum of libraries technologies. I do some part-time help for this group too, and I find these issues especially fascinating – it’s always really illuminating to see how people are using our product, and learn what we can do in future versions to increase productivity and reduce pain.
We’re getting closer to a beta of our first service pack for Visual Studio 8. We managed to put lots of good fixes into this release, including many of the issues with highest vote-count on the MSDN Product Feedback Center.(which recently moved to
http://connect.microsoft.com). Your feedback helped make this one of my favourite service pack experiences. And at least this one is coming out soon after the product shipped.
Planning for Vista UI: Orcas is a Vista-focussed release, and once we’ve completed the development process work described above, we can start the serious work of updating our libraries to do a better job targeting Windows Vista. The VS IDE and MFC both need to know about a raft of new controls, messages, and other Windows elements. If you look back at Windows XP, you’ll see that almost as many controls were added in that version of the operating system, and we didn’t have time to add those when Windows XP shipped (in August 2001, just before we shipped VS7). So we’ve gone back and looked at all the missing controls and UI elements back through Windows XP, so we can try to get all of these into Visual Studio Orcas. This is a big planning job that’s only partly complete, but we’re still working on this as we expect to start implementing these new controls sometime in the next 6 weeks.
Not so work related
What do I do when I’m not writing email or working? Well, this weekend I’m off to the
Pacific Northwest Ballet (I’m a subscriber), and I’m going to finish building a Lego model I bought last week. My fiancée and I have to look at reception venues, and we have a couple of parties at to go to with friends. If I get time between that, I need to work on finishing organisation of the latest iteration of the Puzzle Safari event that I organise with some friends here at Microsoft. And of course, I’ll play squash both days, as I do every day. This weekend is more hectic than most, but that’s still a good sampling of what I spend my time doing.
It’s been fun writing this up. Please send me feedback (
firstname.lastname@example.org), good or bad, so we can adjust and talk more about what you’d like to hear from VC. Also, please feel free to follow up with me if you’re interested in more in-depth information on any of the things above. I’ll use the feedback to consider what I might write to my semi-dormant personal blog, or what to ask others to write about here.
I get lots of mail from customers about various subjects. It’s very welcome, and I encourage you to mail me too. I can’t always respond promptly, but it is always read, and I’m sometimes able to help people a little bit. The features of our next product are shaped in part by the feedback we get from you through channels like this, so do make sure your voice is heard. Get in touch – keep the dialog going!
Thanks for using the product, and for your feedback,
Visual C++ Libraries