{"id":104418,"date":"2020-11-04T07:00:00","date_gmt":"2020-11-04T15:00:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/oldnewthing\/?p=104418"},"modified":"2020-11-04T14:51:33","modified_gmt":"2020-11-04T22:51:33","slug":"20201104-00","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20201104-00\/?p=104418","title":{"rendered":"What is the format of the data in the AudioBuffer memory buffer, and how do I convert it to something else?"},"content":{"rendered":"

The Windows Runtime AudioBuffer class<\/a> represents a buffer of audio data. What is the format of this data?<\/p>\n

The memory buffer you obtain from the AudioBuffer object<\/a> takes the form of an array of audio samples. Each audio sample is a collection of IEEE single-precision floating point numbers which represent a linear range of waveform amplitude from \u22121.0 to +1.0.<\/p>\n

In C#, this floating point format is known as System.<\/code>Single<\/code>. In C++, it is typically represented by float<\/code>.<\/p>\n

Each sample contains one value for each channel, and the channels come in a specified order, described in the documentation for the WAVE\u00adFORMAT\u00adEXTENSIBLE<\/code> structure<\/a>.<\/p>\n

For example, suppose you have a dual-channel audio buffer, say stereo left\/right. The table in the WAVE\u00adFORMAT\u00adEXTENSIBLE<\/code> documentation says that the channels come in the order left<\/i> then right<\/i>. Therefore the values come in this order:<\/p>\n\n\n\n\n\n\n\n\n\n\n
Sample index<\/th>\nChannel<\/th>\nValue index<\/th>\n<\/tr>\n
0<\/td>\nLeft<\/td>\n0<\/td>\n<\/tr>\n
Right<\/td>\n1<\/td>\n<\/tr>\n
1<\/td>\nLeft<\/td>\n2<\/td>\n<\/tr>\n
Right<\/td>\n3<\/td>\n<\/tr>\n
2<\/td>\nLeft<\/td>\n4<\/td>\n<\/tr>\n
Right<\/td>\n5<\/td>\n<\/tr>\n
\u22ee<\/td>\n\u22ee<\/td>\n\u22ee<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Some people call this interleaved format, but whether it’s interleaved depends on what color glasses you’re wearing.<\/p>\n

It’s interleaved if you look at it from the point of view of a channel, since the data from one channel is interleaved with data from the other channels.<\/p>\n

But it’s perfectly linear format if you look at it from the point of view of the samples, since all the data for one sample is packed together.<\/p>\n

Okay, next question: How do you convert this to other formats?<\/p>\n

Well, that depends on what other format you’re converting it to. If you’re converting to a linear format,\u00b9 then you can perform a simple linear conversion. We know that the result is going to be f<\/var>(x) = a<\/var>x<\/var> + b<\/var> for some values of a<\/var> and b<\/var>. We just need to figure out what those values are.<\/p>\n\n\n\n\n
Substitute x<\/var> = \u22121.0:\u2003<\/td>\nv<\/var>min<\/sub> = a<\/var> \u00d7 (\u22121) + b<\/var><\/td>\n<\/tr>\n
Substitute x<\/var> = +1.0:\u2003<\/td>\nv<\/var>max<\/sub> = a<\/var> \u00d7 (+1) + b<\/var><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Solving the system of simultaneous equations gives<\/p>\n\n\n\n\n
a<\/var> = (v<\/var>max<\/sub> \u2212 v<\/var>min<\/sub>)\/2<\/td>\n<\/tr>\n
b<\/var> = (v<\/var>max<\/sub> + v<\/var>min<\/sub>)\/2<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

There is an extra wrinkle to this formula: If the destination is an integer range, then the negative values will have one extra value of range compared to the positive values. For example, a 16-bit signed value will range from \u221232768 to + 32767. If we plug zero into the function, we get just b<\/var>, which is the average between the high and low values, and which will be numerically \u2212\u00bd rather than zero.<\/p>\n

I don’t know how audio people usually solve this problem. One option I’ve seen is to throw out the most negative value, so the effective range for a 16-bit signed value is \u221232767 to +32767. In that case, the formula simplifies to merely multiplying by v<\/var>\u2098\u2090\u2093.<\/p>\n

Maybe some audio people can tell me what they do here.<\/p>\n

\u00b9 There are nonlinear formats for audio data. For example, \u03bc-law is a companding<\/a> algorithm which uses a nonlinear formula in order to express a wider dynamic range in a small space, at a cost of resolution at the extremes.<\/p>\n","protected":false},"excerpt":{"rendered":"

It’s a bunch of floating point samples, in a particular order.<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-104418","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"

It’s a bunch of floating point samples, in a particular order.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/104418","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=104418"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/104418\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=104418"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=104418"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=104418"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}