The updated MIDI 2.0 Specifications have been published

The updated MIDI 2.0 specifications, which were under MIDI Association and AMEI NDA until a week ago, have been published! This is a huge milestone for the MIDI Association and AMEI, and unlocks a lot of public discussion and work. Congratulations to all involved! If I am a bit overzealous with exclamation points in this post, it’s because I’ve been waiting for and working towards this for a long time. 🙂

Updated MIDI 2.0 Specifications

Why were the specifications under NDA? Until specifications are fully adopted by the MIDI Association and AMEI, we keep them inside the associations. During the review and approval process, there are always changes, and sometimes those can be small but significant. Keeping them under NDA until the final vote ensures that we don’t end up with diverging versions of the specification and implementations out in the public and in products. The NDA means that the in-progress specifications can only be shared with voting members of the MIDI Association and AMEI. This really works out in everyone’s best interest, especially the MIDI user.

The updated specifications have some important changes. For example, MIDI-CI protocol negotiation has been deprecated (most other MIDI-CI uses remain), which is a big deal for us because it just didn’t make sense to handle things that way on a Plug & Play aware OS. There is also the concept of “groupless” messages (more correctly referred to as “stream” messages) for function blocks and other configuration, which fundamentally changes how you might interact with a UMP Endpoint. Lots of goodness in there. Function Blocks are a big section in the specification, and worth the read.

Function blocks are similar to the USB Group Terminal blocks, but work regardless of transport, and are more flexible overall. They are discovered within the MIDI protocol rather than through driver or other information. They are not themselves addressable entities, but instead provide context to MIDI groups. In MIDI 1.0 the port (really a virtual cable in the endpoint) is the addressable entity. In MIDI 2.0, the UMP Endpoint is that addressable entity, and the groups (conceptually similar to virtual cables) are in the messages as an address on that UMP Endpoint. Function blocks provide useful metadata about the groups and what they support. Prior to the addition of groupless stream messages, one could argue that breaking it all out into “ports”, with one port per group per direction, was a reasonable thing to do in MIDI 2.0. The UMP Endpoint addressing approach is something all of us in the working groups have come to agreement on over the past year. This is important so you know why the API and SDK include UMP Endpoints, but not input/output ports, and why function blocks and group terminal blocks are metadata. It also helps provide some context to the group<->port mapping we’ll provide to the legacy MIDI 1.0 APIs, and why those APIs cannot access groupless messages.

Through the various working groups, we’ve had a lot of input into these specifications, and are really happy with how they’ve ended up. There were many people involved, but a huge thank you to Mike Kent, Andrew Mee, and of course, Athan Billias for their hard work there. Leading the process of specification authoring and voting, across organizations, is a ton of work — not to mention actually writing it all while we nitpick on terms like “UMP Endpoint” vs “Endpoint” vs “USB Endpoint” 🙂

Note that the USB MIDI 2.0 specification also needs to be updated in a few areas, but hasn’t yet (the USB-IF controls the document, so it has a very different revision process). Instead, the different OS companies and other stakeholders have gotten together in the working groups in the MIDI Association to ensure we’re all creating compatible devices and drivers. If you are considering creating a USB implementation, I highly encourage you to join the MIDI association so you have access to all the discussions and best practices which have evolved.

In the MIDI Association, we’re also working on a developer guide which contains some guidance above and beyond the specifications. It contains things like how to deal with Group Terminal Blocks and Function Blocks when both are present, how multi-client works on the different operating systems, the importance of unique identifiers, naming, and much more.

Beyond that, in the MIDI Association, we’re also quickly converging on a really good Network MIDI 2.0 standard. As ubiquitous as USB MIDI is, I’m especially excited about Network MIDI 2.0 because it has the potential to move us away from length limitations in USB cables, and is simple for devices to implement. Of course, our implementation of that will also be fully open source, so anyone who wants to refer to it or use it on their devices can do so.

If you are creating a commercial MIDI 2.0 product of any sort, you should become a corporate member of the MIDI Association. You’ll have access to the corporate member forums, visibility into in-development standards, the ability to shape MIDI 2.0, and if your product qualifies, the right to use the MIDI logo. You can apply here

Opening the repo

The first thing the publishing of the specifications allows is opening of our repo to the public. As you can imagine, being an open source project, but with a closed repo, was not a great combination. We had invited AMEI and MIDI Association members to it, but it’s really the public community of developers we want this to be visible to.

You can find the repo here, but before you click, I encourage you to read on so you know what you’re looking at/for.

Microsoft MIDI Github Repo

Before you visit the repo, read on so you know what to look at and have the right expectations about where we are in development.

What does this mean?

Opening the repo is the first step in a larger musician community-focused and evergreen implementation of MIDI on Windows. We still have a lot of ground to cover, but by opening the repo, you can join us on the journey, provide feedback, and help us help you.

How open is this project, really?

Microsoft has gotten so much better with integrating and supporting open source over the years. We have internal guidance that encourages openness while protecting our products. We have groups we can ask questions to, and legal advisors who help us make sure we’re always doing the right thing. Nevertheless, creating a truly open source project for something which deeply integrates with Windows, which is not open source as you know, has been a small but welcome challenge for all of us.

By default, we are open source for everything which can be, even going so far as to create clean-room versions of code when there’s any question about what can or can’t be OSS (this has happened with some of the code already, and also BLE MIDI 1.0 will be redone in this way). We’re not open sourcing so that we can dump this on the community (I know that is something the community is always concerned about, and rightly so), but instead so we can engage the developer community, get feedback, enable deep insight into what’s happening, and share MIDI 2.0 with the rest of the world in the most useful way possible.

Of course, we also have signing and HLK/testing/certification requirements which have to happen outside of the repo, but our intent here is to make sure everything in the repo is buildable by you, and if you want to bypass driver signing on your own developer PC, using a supported version of Windows, you can run it all right from the repo.

This project is not being designed explicitly as a cross-platform project (at the interface layers, it has a lot of “Windows” in there like COM and WinRT), but we want to make it as easy as possible to port this in whole or in part to other operating systems should you want to. So no license blockers there, and the implementation is all in the repo. Finally, we want to be as open as possible here so you can see what’s actually happening in the code, and provide bug reports, issues, PRs, and suggestions on how we could do things better.

Although our intent is to make it so creating drivers for new transports is a thing of the past, the USB MIDI 2.0 class driver being OSS will also help companies who do need to create a custom version of the Kernel driver to, for example, synchronize with audio in the same device. If that’s in your plans, talk to me, so we can best support the work.

Creating Kernel drivers is a lot of work, and a real skill. Many of the things C++ developers are used to having available in user mode and simply not available in kernel mode (like most of the standard library, for example). It also brings with it high testing, signing, and compatibility requirements. The new transport plugin model (the service abstractions and settings apps plugins) means that this is no longer necessary for MIDI unless you must have direct access to kernel streaming and similar interfaces. So, for example, a Thunderbolt or PCIe device which doesn’t use the USB class driver may need a custom driver, but virtual, network, Bluetooth, and others can be user-mode and use existing APIs in Windows to communicate with the devices. This makes development a lot easier, especially because the abstractions (service plugin interfaces) can be used during development directly from your test code.

How we are developing

There are different teams involved in creation of the full MIDI 2.0 stack, from AMEI and AmeNote, to Windows Silicon, Graphics & Audio (SiGMa), to my APS team in Windows, and the even MIDI Association itself with its open source libraries, and we each have different development styles, preferred tools, etc. Rather than try to enforce one approach across all contributing teams, as we work towards the initial release and all the infrastructure it involves, we are embracing those tools, processes, and patterns which make us the most productive. For clarity, here are the major items and how they are developed:

Outside contributions will conform to the style for the project they contribute to.

There’s more detail on the planned release mechanisms and more on the main README on Github. Of course, we remain flexible and may change approach at any time in order to best meet the requirements of the project, especially after the initial release.

Finally, we’ve been prototyping a LOT over the past months, so if you find something weird in the repo, just let me know. There may be corners we haven’t yet swept out, but need to.

What about Pull Requests? Issues? Discussion?

Without code you can test, it’s a little early for PRs and Issues, but we welcome them now and in the future. If you are a developer and find issues, we encourage you to use Github for feedback. For users, and for general discussion, please join our Discord Server here:

Windows MIDI Services Discord

In the future, we’ll also have examples, docs, and tools available in the repo to help you get set up to contribute code.

Setting Expectations – What’s there?

To date, we’ve spent a lot of time on prototyping and architecture, and have settled on the approaches we’re using throughout the new Windows MIDI Services. During that time, some key concepts were also changing (perhaps “clarifying” would be more appropriate) in MIDI 2.0, so we also revisited some of our ideas for things like addressing. We also had a ton of work to do to ensure the MIDI 2.0 Class Driver will work on Windows. (As it turns out, our USB Audio 2 class driver claimed MIDI 2.0 compatibility, accidentally. Oops. And also we have more complications due to our MIDI 1.0 and USB Audio 1.0 class drivers being the same driver file.) So now that we have those sorted in internal branches, we can continue on.

Today, the repo does not yet have anything super interesting for folks looking to actually try things out. Lots of stub work, empty calls, etc. I will continue to update everyone as we hit development milestones, especially as we get closer to developer previews this summer and fall, and our first release this year. My latest code is typically in the pete-dev branch, but that will likely may not build. I PR to main when something builds and is worth looking at. We’re not yet at the point where something in the SDK has to pass unit tests to go to main, but we are doing that with the service. For now. treat the entire app SDK as though it has [experimental] on everything.

In addition to the root README, here are some other things you may want to look at (not direct links, just paths on the repo for you to explore as you will)

• The license and our intent behind it FAQ-License-and-Intent.md
• App developer sample code: get-started/midi-developers/app-developers/samples/cpp-winrt/sdk-client-basics
• Programming languages FAQ get-started/midi-developers/FAQ-Programming-Languages.md
• App SDK “shape”. Happy to field questions on this on Discord. src/app-dev-sdk
• WIP version of the Windows Service and abstractions. We’ll have a newer version of this on Github soon. src/service
• WIP version of the USB driver (this is an older rev built for demo use without the service. New version coming soon) src/usb-driver/USB-UMPDriver/USB-UMPDriver

The items in the repo today are items we intend to deliver. We’ve done a LOT of disposable prototyping work over the past months, and have learned a lot in the process. For example, the early Network MIDI 2.0 transport we had at the NAMM show, and early service, API, and SDK ideation and high-performance buffer testing.

Demo created for the NAMM Show and prototyping meet-up with Network MIDI 2.0 in the Windows MIDI Services Settings app. The MIDI Association’s MIDI workbench on the lower left, Windows MIDI Services Settings on the right. Messages communicated across UDP using the Network MIDI 2.0 draft protocol. At the NAMM show we also had the USB MIDI 2.0 driver on display, with MIDI Workbench, communicating to the AmeNote ProtoZOA over USB MIDI 2.0

You can view our Windows MIDI Services feature backlog and milestones in the GitHub project here. This is where you’ll find the important milestones and the big features. I don’t like to micromanage projects, but if there’s something you’re especially interested in which doesn’t show up, let me know through a feature request on GitHub and I will consider adding it to the appropriate milestone.

Repo Practices

The repo is vanilla at the moment, not using much of the new hotness in various areas. We will add CI/CD and automated testing as things stabilize. I’ve been speaking to others on my team to learn some of their best practices (the team I’m on also owns the various Community Toolkit projects and we have a ton of collective OSS experience in my org). But if you see something specific missing that you really think we should do to make the repo more efficient or friendlier to contributors or interested parties, please let me know by opening an issue on GitHub.

What next?

The team is continuing to work on the core infrastructure for Windows MIDI Services, including the Windows service, the RPC calls into the service, the required abstraction layers, kernel streaming integration for drivers, the USB driver itself, and the API which surfaces all that to the SDK and through that to apps. We’re planning several milestones for developer-focused previews with different features enabled.

We have a huge backlog of features we plan to implement, and this project model enables us to do continue to implement and innovate after our first release. We won’t get everything done for day 1, and that is part of the plan. Instead, we want to make sure we are well set up to innovate and iterate on this going forward.

Here’s the current tentative schedule

Again, those dates are tentative, because we are quality driven on this. I know many of you who are, but if you are developing a commercial product and need to snap to specific dates and have not reached out to me, please do so as PM in Discord or through my email pete dot brown at Microsoft dot com. As we hit a release cadence, I’ll post updates on the repo and on Discord.

Were continuing down the path for a first release this year, covering all the core MIDI 2.0 requirements we have from working with software developers, manufacturers, the MIDI Association, AMEI, and more. Our primary focus for the first release is around those core capabilities, excellent performance, and reliability. Backwards compatibility with the existing WinRT and WinMM APIs will follow shortly after the 1.0 release, and the tools will be minimal at first.

Those early releases will be out-of-band shipments until we’re sure the API is in a state that anything else we need to add won’t break compatibility or performance. (We did something similar with WinRT MIDI in the early days where it was a NuGet package until we rolled it into Windows 10.) After that, we’ll start shipping the API and Service in-box with Windows. We expect most of the development will then turn to those pieces which will continue to ship with apps or out of band: the end-user tools, SDK, plugins, etc.

Microsoft MIDI Github Repo

Thanks again to the MIDI Association and to AMEI for their collaboration throughout this project!

And thank you for joining us on this journey. We know that everything about this from spec approval through to implementation has taken quite a bit longer than planned, but we are committed to having an amazing MIDI 2.0 experience on Windows, and improving everything about MIDI 1.0 at the same time.

A few related blog posts

• The importance of including a unique iSerialNumber in your USB MIDI Devices
• The new Windows MIDI Services – Spring 2023 Update
• MIDI @ 40 and the new Windows MIDI Services