Introducing the Azure Quantum Development Kit Preview
100x faster, 100x smaller, and it runs in the browser!
The Azure Quantum team is excited to announce the initial preview of the new Azure Quantum Development Kit (or QDK for short). This has been entirely rebuilt using a new codebase on a new technology stack, and this blog post outlines the why, the how, and some of the benefits of doing so.
The “tl;dr” is that we rewrote it (mostly) in Rust which compiles to WebAssembly for VS Code or the web, and to native binaries for Python. It’s over 100x smaller, over 100x faster, much easier to install & use, works fully in the browser, and is much more productive & fun for the team to work on.
Give it a try via the instructions at https://github.com/microsoft/qsharp/wiki/Installation, and read on for the details…
Our goals
The existing Quantum Development Kit has grown organically over several years, first shipping in late 2017. Being in a fast-evolving space, it naturally evolved quickly too, incorporating many features and technologies along the way.
As we reflected on what we’d like the QDK to be going forward, it was clear some of the technologies and features would be a challenge to bring along, and that a re-write might be the best solution. Some of our goals were:
A simplified user experience
Many quantum developers don’t come from a .NET background, being mostly familiar with Python. However, the existing QDK exposes much of the .NET ecosystem to developers, providing an additional learning curve. Some examples being the MSBuild-based project & build system and NuGet package management. When working with customers on issues, they will sometimes be confused when needing to edit .csproj files, run commands such as “dotnet clean”, or troubleshoot NuGet packages for their Q# projects.
Providing a delightful & simplified experience, from installation to learning to coding to troubleshooting to submitting jobs to quantum computers is our primary goal.
Platform support
The existing QDK has some code and dependencies that are platform specific. While these were not problems initially, as platforms have evolved this has caused challenges. For example, Apple Silicon and Windows on ARM64 are not fully supported in the existing QDK. We also wanted the tools to run in the browser, such as in our new https://quantum.microsoft.com portal, or in a https://vscode.dev hosted editor.
Performance & reliability
With the runtime dependencies in the existing QDK, the full set of binaries that need to be installed has grown quite large. Besides the .NET runtime itself, there are some F# library dependencies in the parser, some C++ multi-threading library dependencies in the simulator, some NuGet dependencies for the Q# project SDK, etc. In total, this can add up to over 180MB when installed locally after building a simple Q# project. Coordinating the download and initialization of the binaries, as well as the complexity of the interactions between them, can often lead to performance & reliability issues.
Engineering velocity
As the existing QDK had come to span multiple repositories, multiple build pipelines, multiple languages & runtimes (each often with their own set of dependencies), and multiple distribution channels, the speed at which we could check in a feature or produce a release has slowed, and a great deal of time is spent on codebase maintenance, security updates, and troubleshooting build issues. To provide a productive (and enjoyable) engineering system going forward, dramatic simplification was needed.
The solution
Around the end of 2022 we set about prototyping some ideas, which grew into the new QDK we are releasing in preview today. The basic philosophy behind engineering the new QDK is as follows:
Write mostly in Rust
By writing as much as possible in Rust, we have a codebase that can easily target native binaries for any platform supported by the Rust compiler (which we build into our Python wheels) and build for WebAssembly (via wasm-bindgen) to run in the browser. With a focused codebase, the resulting binaries are very small & fast too.
Keep technologies and dependencies to a minimum
There is a cost to every dependency you take. The cost to learn it, the cost to install it (i.e., build times and disk space), the cost to update & maintain it (i.e., as security issues are reported), the cost to final product size, and so on. Sometimes these costs are worth paying for what you get in return, but the taxes accumulate over time. We are very mindful and minimal in the dependencies we take.
For our new codebase, we have limited the languages used to:
- Rust for the core of the product. This has the ‘batteries included’ benefit of cargo to manage dependencies, builds, testing, etc.
- Python, as we build & ship packages to PyPI as part of the QDK and use Python for scripting tasks in the repo where practical.
- JavaScript (including TypeScript), as we build a VS Code extension and write some web integration code.
For those three languages, we keep dependencies to a minimum, nearly all of which can be seen in the Cargo.toml and package.json files at the root of the repo.
The below high-level diagram shows roughly how this all fits together in our VS Code extension, Python packages, and for general web site integration.
Simple & fast engineering
Setting up a build environment for developers (or CI agents) should be fast. For the new codebase, currently you just install Rust, Python, and Node.js, clone one repo, and run one Python build script.
Developing the product should be fast. When working on the core compiler Rust code, the development inner-loop is often as fast as clicking ‘run’ on a unit test in VS Code via the excellent “rust-analyzer” extension. When working on the TypeScript code for the VS Code extension, with “esbuild” running in watch-mode it’s as quick as saving the changes and pressing F5 to launch the Extension Development Host.
The build infrastructure should be easy to keep working. Our CI and build pipeline use the same ‘build.py’ script in the root of the repo that developers use locally to build & test.
Keep the product focused
Last but certainly not least, is to avoid the extraneous. Every feature added should have a clear need and add significant value. This provides for a more streamlined & intuitive product for the customer, and a less complex codebase to do further development in.
The result
We’re pretty proud of the result. It’s no exaggeration to say the new Azure Quantum Development Kit is 100x smaller, 100x faster, available on Windows, Mac, Linux, and the web, and is a greatly simplified user experience.
Size
As outlined above, the existing QDK results in over 180MB of binaries locally once a project is fully built and all dependencies installed. The VSIX package for our new VS Code extension is currently around 700KB and includes everything needed for Q# development in VS Code. (If you ‘pip install’ our Python packages to work with Q# via Python, that’s around another 1.3MB). Installation typically takes a couple of seconds with no other dependencies. If you have VS Code, (and Python/Jupyter if desired), you’re ready to install.
Speed
We have examples of programs that would take minutes to compile in the existing QDK. Those same programs are now measured in milliseconds in the new QDK. The language service is so fast, most operations are done on every keystroke and feel instant. The simulator can run 1000s of ‘shots’ per second for many common algorithms on a good laptop.
The build pipelines for the existing QDK take between 2 – 3 hours to complete, are fragile, and issues often require coordinated check-ins across multiple repos. For the new QDK, all code is in one repo, and we build, test, and push live to our online playground in around 10 mins on every commit to main. Our publishing pipeline uses largely the same script.
We’ve built an extremely fast & reliable installation, language service, compiler, and debugger. Oh, and it all works inside the browser too!
VS Code for the Web
A couple of years ago VS Code introduced VS Code for the Web (https://code.visualstudio.com/docs/editor/vscode-web), with the ability to run the IDE in a browser with no local install, such as at https://vscode.dev or by pressing “.” when in a GitHub repo. By building our extension entirely as a web extension ALL our features run equally well in VS Code desktop or in the browser.
By way of example, the below screenshot shows loading the editor in the browser by visiting https://vscode.dev, running a Q# file under the debugger, viewing the quantum simulator output in the Debug Console, while also signed in to an Azure Quantum Workspace shown in the Explorer sidebar (to which the current program could be submitted) – all without anything needing to be installed on the local machine.
We think the improvements in the user experience for the new QDK really are a quantum leap (bad pun intended).
What’s ahead
This is an early preview, and we still have several features to add before we get to our ‘stable’ release, some of the main ones being:
- Multi-file support: For this preview all code for a Q# program needs to be in one source file. (With Q#, you can simply ‘concat’ source files together if need be).
- Richer QIR support: This preview currently can compile programs for hardware that supports the QIR base-profile which, as the name suggests, provides for a basic level of capabilities. With some hardware starting to support more advanced capabilities (currently being specified in the QIR Adaptive Profile), we will be adding support for that also. (Note that running in the simulator isn’t restricted to these profiles and can run any Q# code).
- Migration: Being not entirely backwards compatible with the existing QDK, we also have a lot of work to do on updating samples & documentation. (The “Differences from the previous QDK” page on our wiki will highlight changes and how to migrate code).
Once the core product is solid, we have a laundry list of further features and Q# language improvements we want to get to, which you can view and contribute to on our GitHub repo.
The existing QDK (https://learn.microsoft.com/en-us/azure/quantum/install-overview-qdk) is still fully supported and should be used if the new QDK Preview doesn’t meet your needs or is changing too frequently as we iterate towards our stable release.
Get involved!
We’d love for you to give it a try and give us your feedback. The installation guide and other getting started documentation is currently on our GitHub wiki at https://github.com/microsoft/qsharp/wiki/Installation. You can report any issues, weigh in on feature requests, or contribute code on that same GitHub repo.
Light
Dark
6 comments
Ohh, always love a good RIIR story! I’ve just started playing with writing a language server in Rust for VSCode, sounds like you all implemented it very similar to how I was thinking of doing it with WASM in the browser for the server. I’ll have to go take a look at it if the source is available.
As an employee of the same company, you don’t report .net issues to your colleagues to help better improve the company’s products, but instead tout other external tools.
I don’t think this is a good approach. It also suggests Microsoft may lack cohesion.
At the same time, the C# language developed by the .net team is not popular when promoted internally. This shows that the development of .net in recent years is not good enough. It should accept widespread criticism, improve and optimize its products. At the same time, The lack of some Geek-type characters among the senior executives of the .net team has resulted in the current development of C# being very mediocre, and these should be taken seriously.
The prior QDK implementation was slow and large not because of .NET, but because it had accumulated too much cruft and dependencies over time, as I tried to outline.
The choice of Rust was primarily due to several factors, namely:
– Our quantum tooling is focused on QIR as an intermediate representation (outlined in the article), which is an LLVM format. As such using other tooling based on LLVM, such as Rust, has its benefits over choosing a non-LLVM toolchain.
– Likewise, several tools in the QIR ecosystem are already in Rust. For example, the simulator we take a dependency on is written in Rust (see https://github.com/qir-alliance/qir-runner), thus integration with this package is simpler if also using Rust.
– Rust has several well-maintained crates we use to generate the bindings needed to expose the core compiler as Python packages (Maturin and Py03) or WebAssembly (wasm-bindgen and wasm-pack). This again saves us much in the way of integration work for the environments we target.
– While .NET 8 is making great advances in AOT compilation and will ship in a couple months, we started this work nearly a year ago, and .NET 7 doesn’t have support for all the platforms we target (e.g. Apple Silicon), and that was a requirement for us.
So again, it’s not that .NET can’t be fast & small and do a great job for much of what we need, but the ecosystem around Rust gave us a lot of things for free that this particular project had as requirements.
(Note: I did have much of this reasoning in the original blog draft, but as it was already running to 7 pages, figured I should trim it down. Maybe I should go back and add these details in).
Dropping .net for Javascript, Rust or Python, seem to be a sign of immature capabilities, and keeping with experimental tools.
Is Q# a language or a fancy facade to interpreters spaghetti code mixing inputs. Maybe Quantum computing is not ready for a real language yet….
If even Microsoft doesn’t use .NET for most of its tools and is abandoning it, why should people use it?
You should use .NET everywhere, give feedback to the team and ensure that .NET improves. Instead of doing this, you abandon .NET and turn to popular options. It’s not nice.
If you do this, other people will do the same. By doing this, you cause the usage rate of .NET to decrease. If it continues like this, the usage percentage will drop significantly after 5-10 years and it will remain a niche in some corporate companies. You are doing wrong.
We are certainly not abandoning it, and still use C#/.NET heavily in much of our work in Azure Quantum. I’ve outlined the reasons for the choice based on the requirements of this particular project in the comment above at https://devblogs.microsoft.com/qsharp/introducing-the-azure-quantum-development-kit-preview/#comment-204 .