{"id":514,"date":"2023-06-06T15:53:49","date_gmt":"2023-06-06T22:53:49","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/windows-music-dev\/?p=514"},"modified":"2023-06-06T15:53:49","modified_gmt":"2023-06-06T22:53:49","slug":"the-importance-of-including-a-unique-iserialnumber-in-your-usb-midi-devices","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/windows-music-dev\/the-importance-of-including-a-unique-iserialnumber-in-your-usb-midi-devices\/","title":{"rendered":"The importance of including a unique iSerialNumber in your USB MIDI Devices"},"content":{"rendered":"<h2>The importance of including a unique iSerialNumber in your USB MIDI Devices<\/h2>\n<p>Uniquely identifying connected devices is something all operating systems must do in order to function. But persistently identifying them, across moves, reboots, and more, can be more challenging. This post covers why this topic is so important to Windows MIDI Services and what manufacturers and customers can expect to see in Windows MIDI Services.<\/p>\n<blockquote>\n<p>Please note that the scope of this post is just USB MIDI devices. Other devices, like storage, have different requirements and different approaches to identification.<\/p>\n<\/blockquote>\n<p>TLDR; Please provide a valid and unique <code>iSerialNumber<\/code> string descriptor for your USB devices. If you cannot provide that value, do not provide any string descriptor for that (set the <code>iSerialNumber<\/code> field value to <code>0x00<\/code> so it does not point to any string descriptor). If you want to learn more, please read on.<\/p>\n<blockquote>\n<p>Note: When I mention the <code>iSerialNumber<\/code> field in the rest of this post, I&#8217;m generally referring to the string descriptor the <code>iSerialNumber<\/code> field points to.<\/p>\n<\/blockquote>\n<h2>How Windows identifies USB devices<\/h2>\n<p><strong>Have you ever noticed that when you unplug a device from one USB port, and then plug it into a different USB port, you&#8217;ll sometimes see it pop up as a new device, and maybe even get a driver installed?<\/strong><\/p>\n<p>That can happen when a USB device doesn&#8217;t include the <code>iSerialNumber<\/code> field, and you plug it into a port which shows up as a different parent device, impacting the calculated unique id for that USB device. When a device doesn&#8217;t include an <code>iSerialNumber<\/code>, operating systems use various techniques to uniquely identify the device, as best as they can.<\/p>\n<blockquote>\n<p>Although implementations vary, this process isn&#8217;t unique to Windows. macOS, Linux, and other operating systems have similar algorithms to try to smartly identify a device in the absence of a serial number. They also have similar limitations.<\/p>\n<\/blockquote>\n<p>Here are some low-level details about how devices are uniquely identified in Windows. This is quoted from our MS Learn docs (links below):<\/p>\n<blockquote>\n<p>&#8230; For BusQueryInstanceID, a bus driver should supply a string that contains the instance ID for the device. Setup and bus drivers use the instance ID, with other information, to differentiate between two identical devices on the computer. The instance ID is either unique across the whole computer or just unique on the device&#8217;s parent bus.<\/p>\n<p>If an instance ID is only unique on the bus, the bus driver specifies that string for <code>BusQueryInstanceID<\/code> but also specifies a <code>UniqueID<\/code> value of FALSE in response to an <code>IRP_MN_QUERY_CAPABILITIES<\/code> request for the device. If <code>UniqueID<\/code> is FALSE, the PnP manager enhances the instance ID by adding information about the device&#8217;s parent and thus makes the ID unique on the computer. In this case the bus driver should not take extra steps to make its devices&#8217; instance IDs globally unique; just return the appropriate capabilities information and the operating system takes care of it.<\/p>\n<p>If a bus driver can supply a globally unique ID for each child device, such as a serial number, the bus driver specifies those strings for <code>BusQueryInstanceID<\/code> and specifies a <code>UniqueID<\/code> value of TRUE in response to an <code>IRP_MN_QUERY_CAPABILITIES<\/code> request for each device. &#8230;<\/p>\n<\/blockquote>\n<p><strong>So you can see in the above description, that if an <code>iSerialNumber<\/code> is not supplied, Windows will use the device&#8217;s parent information as part of the differentiator to uniquely identify the device. If a device is moved to a new parent (plugged into a different port, plugged into a USB hub, etc.), that id changes and the device appears as a new device. Any link to its past is effectively lost.<\/strong><\/p>\n<p>Just as a reminder, storage and some other device classes have additional mechanisms for unique identification (a storage device can, well, store a supplied id). We&#8217;re talking about MIDI devices here.<\/p>\n<p>References<\/p>\n<ul>\n<li><a href=\"https:\/\/learn.microsoft.com\/windows-hardware\/drivers\/kernel\/irp-mn-query-id\">https:\/\/learn.microsoft.com\/windows-hardware\/drivers\/kernel\/irp-mn-query-id<\/a><\/li>\n<li><a href=\"https:\/\/learn.microsoft.com\/windows-hardware\/drivers\/install\/device-instance-ids\">https:\/\/learn.microsoft.com\/windows-hardware\/drivers\/install\/device-instance-ids<\/a><\/li>\n<\/ul>\n<h3>How to check your own devices<\/h3>\n<p>As a developer, or a curious customer, you can check connected devices to see what they report using a number of free tools.<\/p>\n<h4>Plug &amp; Play utility<\/h4>\n<p>First is the command-line tool <code>pnputil<\/code>, which is included with Windows. If you are on recent Windows 11 releases, there&#8217;s a <code>\/properties<\/code> flag which will display additional information about devices, including the full Id of the device.<\/p>\n<p>To list all connected USB devices, and all their properties, here&#8217;s the command line<\/p>\n<pre><code class=\"language-dos\">&gt;pnputil \/enum-devices \/properties \/connected \/bus USB<\/code><\/pre>\n<p>Here&#8217;s some pnputil information from the Arturia Keystep 32 I have here.<\/p>\n<pre><code class=\"language-dos\">DEVPKEY_Device_Capabilities [UINT32]:\n        0x00000094 (148)\nDEVPKEY_Device_InstanceId [String]:\n        USB\\VID_1C75&PID_0288\\00000000001A\nDEVPKEY_Device_BusReportedDeviceDesc [String]:\n        Arturia KeyStep 32\nDEVPKEY_NAME [String]:\n        USB Composite Device<\/code><\/pre>\n<p>And here&#8217;s a Modal Skulpt Synth. I don&#8217;t have more than one Skulpt synth handy to test, but it seems likely that they&#8217;ve hard-coded the iSerialNumber to just &#8220;ffff&#8221;, which is not helpful.<\/p>\n<pre><code class=\"language-dos\">DEVPKEY_Device_Capabilities [UINT32]:\n        0x000000B4 (180)\nDEVPKEY_Device_InstanceId [String]:\n        USB\\VID_04D8&PID_EEFE\\ffff\nDEVPKEY_Device_FriendlyName [String]:\n        Skulpt Synth\nDEVPKEY_NAME [String]:\n        Skulpt Synth<\/code><\/pre>\n<p>In this, we can see the Device Capabilities flag shows that the device has declared CM_DEVCAP_REMOVABLE, CM_DEVCAP_SILENTINSTALL, CM_DEVCAP_SURPRISEREMOVALOK, and importantly, we&#8217;ve set CM_DEVCAP_UNIQUEID because an <code>iSerialNumber<\/code> was provided. The actual values can be found in cfgmgr32.h:<\/p>\n<pre><code class=\"language-cpp\">#define CM_DEVCAP_LOCKSUPPORTED     (0x00000001)\n#define CM_DEVCAP_EJECTSUPPORTED    (0x00000002)\n#define CM_DEVCAP_REMOVABLE         (0x00000004)\n#define CM_DEVCAP_DOCKDEVICE        (0x00000008)\n#define CM_DEVCAP_UNIQUEID          (0x00000010)\n#define CM_DEVCAP_SILENTINSTALL     (0x00000020)\n#define CM_DEVCAP_RAWDEVICEOK       (0x00000040)\n#define CM_DEVCAP_SURPRISEREMOVALOK (0x00000080)\n#define CM_DEVCAP_HARDWAREDISABLED  (0x00000100)\n#define CM_DEVCAP_NONDYNAMIC        (0x00000200)<\/code><\/pre>\n<p>If a device is going to declare that it has a unique ID by providing data in the <code>iSerialNumber<\/code> string, it needs to provide an <code>iSerialNumber<\/code> that is unique amongst other instances of the same device. Again, I can&#8217;t verify that <code>\"ffff\"<\/code> is or is not the same across all the devices, but it looks pretty sus. <strong>Modal, if you&#8217;re reading this, and I got it wrong, please accept my apologies. You know I love your synths. \ud83d\ude42<\/strong><\/p>\n<p>Quite a bit more information is reported, but I won&#8217;t paste it all in here. Try out <code>pnputil<\/code>, it comes with every recent version of Windows.<\/p>\n<h4>USB viewer<\/h4>\n<p>PnP Util provides a view of what Windows sees for a device. Another tool, USB Viewer, shows more detail about what the device reports, and how the descriptors are structured. It&#8217;s free with the Windows SDK, or available <a href=\"https:\/\/github.com\/Microsoft\/Windows-driver-samples\/tree\/main\/usb\/usbview\">on GitHub<\/a>. <a href=\"https:\/\/learn.microsoft.com\/windows-hardware\/drivers\/debugger\/usbview\">You can find information about it here<\/a>.<\/p>\n<p>If you don&#8217;t have the SDK, or don&#8217;t want to use the version on GitHub, or simply want something with more reporting, there&#8217;s an enhanced version of the USB Treeview app created by a third-party. This is not an official endorsement, but this is the version I personally use. <a href=\"https:\/\/www.uwe-sieber.de\/usbtreeview_e.html#download\">It&#8217;s available here.<\/a><\/p>\n<p>Here&#8217;s a segment of the USB Tree Viewer output showing the Modal Skulpt<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/windows-music-dev\/wp-content\/uploads\/sites\/63\/2023\/06\/skulpt-tree-view-01.png\" alt=\"Modal Skulpt USB Tree View\" \/><\/p>\n<p>And here&#8217;s a device (MOTU Express 128 MIDI Interface) which does not implement <code>iSerialNumber<\/code>. Note that it has a <code>0x00<\/code> index for that field, as is proper.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/windows-music-dev\/wp-content\/uploads\/sites\/63\/2023\/06\/motu-tree-view-01.png\" alt=\"MOTU Express 128 USB Tree View\" \/><\/p>\n<p>Finally, here&#8217;s a good implementation of <code>iSerialNumber<\/code>, from the PreSonus ATOM.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/windows-music-dev\/wp-content\/uploads\/sites\/63\/2023\/06\/presonus-tree-view-01.png\" alt=\"PreSonus ATOM USB Tree View\" \/><\/p>\n<p>You can use either of these tools to check to see what your device is reporting to Windows, and how Windows is interpreting what it sees.<\/p>\n<h2>Windows MIDI Services Features that work best with unique ids<\/h2>\n<p>Now we know how devices are identified, and how to verify that information, but how does this impact Windows MIDI Services?<\/p>\n<p>The first defined and approved transport for MIDI 2.0 is USB. There&#8217;s a new device class specification, and Windows also has a new USB MIDI Class 2 driver to support it. There are also other transports in development, with Network MIDI 2.0 the most likely one to hit next. That transport also includes a way to uniquely identify devices, as does the MIDI 2.0 protocol itself. But those are in-protocol mechanisms which, although we will take advantage of them in Windows MIDI Services and use them in much the same way for MIDI, are not at the same level as Kernel-level PnP information from a driver.<\/p>\n<p>The new Windows MIDI Services has a number of planned features which work best when we can uniquely identify a MIDI Device, no matter which USB port it&#8217;s connected on the PC. Not all of these features will be available on day 1, but we will be rolling them out over time as we&#8217;re sure the experience is what you want. Here are two of the more important ones:<\/p>\n<h3>Renaming MIDI Devices \/ UMP Endpoints<\/h3>\n<p>One of the most requested features for Windows MIDI Services is the ability to rename devices \/ UMP Endpoints. For this to be meaningful, we need to be able to uniquely identify the MIDI device you want to rename.<\/p>\n<p>Renaming a device isn&#8217;t a ton of effort for a user, but it would be super confusing if they had two of the same synth, and one was set up with bass patches, and the other with pads, and Windows got confused about which one was which, and applied the wrong names, after they were unplugged and brought to a performance.<\/p>\n<h3>User-supplied metadata<\/h3>\n<p>We&#8217;re looking at ways to provide MIDI 2.0 profiles and properties for devices which don&#8217;t natively support those features. That would enable you to, for example, indicate that the synthesizer you have fits the organ profile and can be discovered as such, or that that Piano module you have can conform to the Piano profile.<\/p>\n<p>There can be quite a bit of work involved in supplying this metadata, so we want to make it worth the customer&#8217;s time, and not have any confusion over which connected device the metadata applies to.<\/p>\n<h2>Some scenarios<\/h2>\n<p>There are some scenarios where calculated vs provided unique identifiers really come into play. We will do our best to optimize for a good user experience here, but valid <code>iSerialNumber<\/code> values will really help everyone.<\/p>\n<h3>MIDI Device connected to PC and never moved or unplugged<\/h3>\n<p>The generated identifiers persist across reboots, so there are no immediate issues here, until devices are moved around, put on a hub, etc. We don&#8217;t want customers to be afraid to unplug and rearrange their studio.<\/p>\n<p>If the device implements <code>iSerialNumber<\/code> properly, then no problem. If it does not, we generate an identifier, which is likely to change especially if you plug it into a different port, or into a hub.<\/p>\n<h3>Setting up that live performance<\/h3>\n<p>This is a more common problem. Let&#8217;s say you have a couple controllers and synthesizers you bring to your performance. If you don&#8217;t plug those devices into the same ports they were in when you practiced at home, especially if you have multiples of any one device, there&#8217;s potential for the OS to get them mixed up. Without an <code>iSerialNumber<\/code>, there&#8217;s no guarantee that the generated identifier will persist or will be applied to the same device.<\/p>\n<p>Many club performers get little to no time to set up their equipment. They need to rush up on stage while the last performer is wrapping up so there&#8217;s no gap between the previous performance and their performance. It&#8217;s not like the roadies get there the day before and get everything set up and configured properly in advance. They shouldn&#8217;t need to struggle with recreating their performance setup, in the dark, while everyone is staring at them. This is just stress a live performer doesn&#8217;t need when setting up. <strong>And as a device manufacturer, you want to encourage your customers to buy more than one of your MIDI device, right? \ud83d\ude42<\/strong><\/p>\n<h2>Some options for iSerialNumber<\/h2>\n<p>The USB <code>iSerialNumber<\/code> field is a pointer to a text record, just like <code>iManufacturer<\/code> and other USB strings. There are practical limits on how long the string can be. The entire Windows Device Instance Id, which includes other information in addition to the serial, needs to be <code>MAX_DEVICE_ID_LEN<\/code> characters or less &#8212; currently 200 characters per cfgmgr32.h. This means you should keep your number to something in the range of 32 alphanumeric digits. It can be larger (quite a bit, as long as you check that the Device Instance Id is still valid), but 32 digits should be plenty for this use.<\/p>\n<pre><code class=\"language-cpp\">#define MAX_DEVICE_ID_LEN     200\n#define MAX_DEVNODE_ID_LEN    MAX_DEVICE_ID_LEN<\/code><\/pre>\n<blockquote>\n<p>MIDI 2.0 allows for <code>ProductInstanceId<\/code> values, the in-protocol <code>iSerialNumber<\/code> equivalent here, to be up to 255 characters in length. But the practical limit here is much lower if you want to support Windows devices. We recommend following the same rules for both iSerialNumber and the <code>ProductInstanceId<\/code>, and keeping them in sync when you provide both.<\/p>\n<\/blockquote>\n<p>When I recently met with manufacturers, one revelation they had was that the <strong><code>iSerialNumber<\/code> doesn&#8217;t need to match the serial number on the box; it just needs to be a unique id.<\/strong> Understandably, manufacturers were concerned that the serial number matching the box would add a costly step to the manufacturing process. When I pointed out that we just need something that will uniquely identify a device within the domain of devices in a customer&#8217;s setup, the problem became much simpler to solve.<\/p>\n<p>There&#8217;s no prescription here, as each device manufacturer has their own unique circumstances. However, here are some ideas for how to generate this number.<\/p>\n<table>\n<thead>\n<tr>\n<th>Value<\/th>\n<th>Approach<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>USB controller-provided Id<\/td>\n<td>Some embedded USB \/ Serial converter ICs also include support for <code>iSerialNumber<\/code>, and a small amount of memory to persist that value in<\/td>\n<\/tr>\n<tr>\n<td>Processor Unique Id<\/td>\n<td>Some processors have unique Ids which can be read by code on the device. Simply report this number.<\/td>\n<\/tr>\n<tr>\n<td>MAC Address<\/td>\n<td>If the device has a network port, the MAC address (with special characters removed), or a hash of it, will work as a unique Id here<\/td>\n<\/tr>\n<tr>\n<td>GUID\/UUID<\/td>\n<td>There are established algorithms for generating GUID\/UUID values. In some cases, this is part of the libraries included with microcontrollers. Strip out the special characters, and you end up with a 32 character unique Id that has a very low chance of colliding with any other in the world.<\/td>\n<\/tr>\n<tr>\n<td>Random Number<\/td>\n<td>Random number generators are included with almost all microcontroller libraries. Generating something reasonable like a 10 digit number would be unique enough for use here.<\/td>\n<\/tr>\n<tr>\n<td>User-supplied<\/td>\n<td>If your device has a user interface for entering data (a web server or touch-screen display, for examples), you may also consider allowing the customer to provide\/change the serial number themselves<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>When we were discussing random number generators, one developer started going through how common collisions are with UUIDs. That&#8217;s mostly a non-issue here. The number doesn&#8217;t need to be unique in the world, just unique within the domain of the devices he user has connected to their computer. If you use a sufficiently large domain of numbers to draw from, you&#8217;ll be fine.<\/p>\n<p>When creating the id, we <del>recommend<\/del> require removing any special characters, and leave it to just letters and numbers. Specifically, character values less than or equal to <code>0x20<\/code> (space), greater than <code>0x7F<\/code>, or equal to <code>0x2C<\/code> (comma) are <strong>not allowed<\/strong> unless you really want to bugcheck the PC. You can find more information in the <a href=\"https:\/\/learn.microsoft.com\/windows-hardware\/drivers\/kernel\/irp-mn-query-id\">IRP_MN_QUERY_ID kernel driver docs<\/a>.<\/p>\n<h2>What happens when you can&#8217;t support a serial number?<\/h2>\n<p>There are cases when a device cannot support an <code>iSerialNumber<\/code> because it has no available persistent storage for this. That is far less common these days, especially with devices which can support MIDI 2.0, but it does happen. In addition, the body of devices already available without an <code>iSerialNumber<\/code> need to continue to function in Windows without a degraded experience.<\/p>\n<p><strong>Excellent user experience is top priority for Windows MIDI Services.<\/strong><\/p>\n<p>At a minimum, everything will continue using the algorithms the Windows USB stack uses today. <strong>In the tools, we may warn the user that the device does not support a serial number so they can consider how much time\/effort they want to invest in customization.<\/strong> We&#8217;re optimizing for the user experience here, and investing time in changing names, adding metadata, etc. only to have it disassociated from the device, or applied to another instance of the device, would not be good there.<\/p>\n<p>In addition, in the absence of <code>iSerialNumber<\/code>, we are working on identifying MIDI devices by other properties reported by the driver. For example, the device name as reported by the hardware (although device naming is an entirely different and convoluted discussion). This can work if you only own one of those devices, but does not work well when you have more than one of the devices connected at a time, or more than one of the device with other unique values (like patch sets on a synth or sound module, or firmware-stored CC mapping on controllers).<\/p>\n<h2>Our request for device manufacturers and makers<\/h2>\n<p>Including and reporting <code>iSerialNumber<\/code> on your USB devices will improve the user experience across all operating systems, not just Windows, so please considering this as part of your device firmware and\/or provisioning.<\/p>\n<p>Did I say please? PLEASE properly implement <code>iSerialNumber<\/code>. Your customers will love you for it, even if they don&#8217;t quite realize why \ud83d\ude42<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The Importance of iSerialNumber in your USB MIDI Devices, especially in the new Windows MIDI Services<\/p>\n","protected":false},"author":38761,"featured_media":515,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[33,2],"tags":[30,5,29,28],"class_list":["post-514","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-hardware","category-midi","tag-api","tag-midi","tag-midi-1-0","tag-midi-2-0"],"acf":[],"blog_post_summary":"<p>The Importance of iSerialNumber in your USB MIDI Devices, especially in the new Windows MIDI Services<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/windows-music-dev\/wp-json\/wp\/v2\/posts\/514","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/windows-music-dev\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/windows-music-dev\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/windows-music-dev\/wp-json\/wp\/v2\/users\/38761"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/windows-music-dev\/wp-json\/wp\/v2\/comments?post=514"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/windows-music-dev\/wp-json\/wp\/v2\/posts\/514\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/windows-music-dev\/wp-json\/wp\/v2\/media\/515"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/windows-music-dev\/wp-json\/wp\/v2\/media?parent=514"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/windows-music-dev\/wp-json\/wp\/v2\/categories?post=514"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/windows-music-dev\/wp-json\/wp\/v2\/tags?post=514"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}