{"id":29542,"date":"2020-08-31T10:00:19","date_gmt":"2020-08-31T17:00:19","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/dotnet\/?p=29542"},"modified":"2020-08-31T13:55:30","modified_gmt":"2020-08-31T20:55:30","slug":"introducing-the-half-type","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/introducing-the-half-type\/","title":{"rendered":"Introducing the Half type!"},"content":{"rendered":"

The IEEE 754<\/code> specification defines many floating point types, including: binary16<\/code>, binary32<\/code>, binary64<\/code> and binary128<\/code>. Most developers are familiar with binary32<\/code> (equivalent to float<\/code> in C#) and binary64<\/code> (equivalent to double<\/code> in C#). They provide a standard format to represent a wide range of values with a precision acceptable for many applications. .NET has always had float<\/code> and double<\/code> and with .NET 5 Preview 7, we’ve added a new Half<\/code> type (equivalent to binary16<\/code>)!<\/p>\n

A Half<\/code> is a binary floating-point number that occupies 16 bits. With half the number of bits as float, a Half<\/code> number can represent values in the range \u00b165504. More formally, the Half<\/code> type is defined as a base-2 16-bit interchange format meant to support the exchange of floating-point data between implementations. One of the primary use cases of the Half<\/code> type is to save on storage space where the computed result does not need to be stored with full precision. Many computation workloads already take advantage of the Half<\/code> type: machine learning, graphics cards, the latest processors, native SIMD libraries etc. With the new Half<\/code> type, we expect to unlock many applications in these workloads.<\/p>\n

Let’s explore the Half<\/code> type:<\/h3>\n

The 16 bits in the Half<\/code> type are split into:<\/p>\n

    \n
  1. Sign bit: 1 bit<\/li>\n
  2. Exponent bits: 5 bits<\/li>\n
  3. Significand bits: 10 bits (with 1 implicit bit that is not stored)<\/li>\n<\/ol>\n

    Despite that fact that the significand is made up of 10 bits, the total precision is really 11 bits. The format is assumed to have an implicit leading bit of value 1 (unless the exponent field is all zeros, in which case the leading bit has a value 0). To represent the number 1 in the Half<\/code> format, we’d use the bits:<\/p>\n

    0\u00a001111\u00a00000000000\u00a0=\u00a01<\/pre>\n
    \n
    The leading bit (our sign bit) is 0<\/code>, indicating a positive number. The exponent bits are 01111<\/code>, or 15<\/code> in decimal. However, the exponent bits don’t represent the exponent directly. Instead, an exponent bias is defined that lets the format represent both positive and negative exponents. For the Half<\/code> type, that exponent bias is 15<\/code>. The true exponent is derived by subtracting 15<\/code> from the stored exponent. Therefore, 01111<\/code> represents the exponent e = 01111 (in binary) - 15 (the exponent bias) = 0<\/code>. The significand is 0000000000<\/code>, which can be interpreted as the number .significand(in base 2)<\/code> in base 2, 0<\/code> in our case. If, for example, the significand was 0000011010 (26 in decimal)<\/code>, we can divide its decimal value 26<\/code> by the number of values representable in 10 bits (1 << 10)<\/code>: so the significand 0000011010 (in binary)<\/code> is 26 \/ (1 << 10) = 26 \/ 1024 = 0.025390625<\/code> in decimal. Finally, because our stored exponent bits (01111)<\/code> are not all 0<\/code>, we have an implicit leading bit of 1<\/code>. Therefore,<\/p>\n
    \n
    0 01111 0000000000 = 2^0 * (1 + 0\/1024) = 1<\/pre>\n<\/div>\n<\/div>\n
    \n
    \n
    In\u00a0general,\u00a0the\u00a016\u00a0bits\u00a0of\u00a0a\u00a0Half<\/code>\u00a0value\u00a0are\u00a0interpreted\u00a0as\u00a0-1^(sign\u00a0bit)\u00a0*\u00a02^(storedExponent\u00a0-\u00a015)\u00a0*\u00a0(implicitBit\u00a0+\u00a0(significand\/1024))<\/code>.\u00a0A\u00a0special\u00a0case\u00a0exists\u00a0for\u00a0the\u00a0stored\u00a0exponent\u00a000000<\/code>.\u00a0In\u00a0this\u00a0case,\u00a0the\u00a0bits\u00a0are\u00a0interpreted\u00a0as\u00a0-1^(sign\u00a0bit)\u00a0*\u00a02^(-14)\u00a0*\u00a0(0\u00a0+\u00a0(significand\/1024))<\/code>.\u00a0Let’s\u00a0look\u00a0at\u00a0the\u00a0bit\u00a0representations of some other numbers in the Half<\/code> format:<\/div>\n<\/div>\n<\/div>\n
    \n
    \n
    <\/h3>\n
    Smallest positive non-zero value<\/h3>\n
    \n
    \n
    <\/code><\/code><\/p>\n
    0\u00a000000\u00a00000000001\u00a0=\u00a0-1^(0)\u00a0*\u00a02^(-14)\u00a0*\u00a0(0\u00a0+\u00a01\/1024)\u00a0\u2248\u00a00.000000059604645<\/pre>\n
    \n<\/code>\u00a0(Note\u00a0the\u00a0implicit\u00a0bit\u00a0is\u00a00\u00a0here\u00a0because\u00a0the\u00a0stored\u00a0exponents\u00a0bits\u00a0are\u00a0all\u00a00)<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
    <\/div>\n
    \n
    \n
    Largest\u00a0normal\u00a0number<\/h3>\n<\/div>\n<\/div>\n
    \n
    0\u00a011110\u00a01111111111\u00a0=\u00a0-1^(0)\u00a0*\u00a02^(15)\u00a0*\u00a0(1\u00a0+\u00a01023\/1024)\u00a0\u2248\u00a065504<\/pre>\n<\/div>\n
    <\/div>\n
    \n
    Negative\u00a0Infinity<\/h3>\n
    \n
    1 11111 0000000000 = -Infinity<\/pre>\n<\/div>\n<\/div>\n
    <\/div>\n
    \n
    A\u00a0peculiarity\u00a0of\u00a0the\u00a0format\u00a0is\u00a0that\u00a0it\u00a0defines\u00a0both\u00a0positive\u00a0and\u00a0negative\u00a00:<\/div>\n
    \n
    1\u00a000000\u00a00000000000\u00a0=\u00a0-0<\/pre>\n<\/div>\n
    <\/div>\n
    \n
    0\u00a000000\u00a00000000000\u00a0=\u00a0+0<\/pre>\n<\/div>\n
    \n
    \n
    Conversions\u00a0to\/from\u00a0float\/double<\/h3>\n
    \n
    A\u00a0Half<\/code>\u00a0can\u00a0be\u00a0converted\u00a0to\/from\u00a0a\u00a0float\/double\u00a0by\u00a0simply\u00a0casting\u00a0it:<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
    \n
    float f = (float)half<\/code>; Half<\/code> h = (Half<\/code>)floatValue;<\/pre>\n
    \n
    \n
    Any Half<\/code> value, because Half<\/code> uses only 16 bits, can be represented as a float\/double<\/code> without loss of precision. However, the inverse is not true. Some precision may be lost when going from float\/double<\/code> to Half<\/code>. In .NET 5.0, the Half<\/code> type is primarily an interchange type with no arithmetic operators defined on it. It only supports parsing, formatting and comparison operators. All arithmetic operations will need an explicit conversion to a float\/double<\/code>. Future versions will consider adding arithmetic operators directly on Half<\/code>.<\/p>\n<\/div>\n<\/div>\n<\/div>\n
    \n
    \n
    \n
    As library authors, one of the points to consider is that a language can add support for a type in the future. It is conceivable that C# adds a half<\/code> type in the future. Language support would enable an identifier such as f16<\/code>(similar to the f<\/code> that exists today) and implicit\/explicit conversions. Thus, the library defined type Half<\/code> needs to be defined in a manner that does not result in any breaking changes if half<\/code> becomes a reality. Specifically, we needed to be careful about adding operators to the Half<\/code> type. Implicit conversions to float\/double<\/code> could lead to potential breaking changes if language support is added. On the other hand, having a Float\/Double<\/code> property on the Half<\/code> type felt less than ideal. In the end, we decided to add explicit operators to convert to\/from float\/double<\/code>. If C# does add support for half<\/code>, no user code would break, since all casts would be explicit.<\/p>\n<\/div>\n<\/div>\n<\/div>\n
    \n
    \n
    Adoption<\/h3>\n
    \n
    We expect that Half<\/code> will find its way into many codebases. The Half<\/code> type plugs a gap in the .NET ecosystem and we expect many numerics libraries to take advantage of it. In the open source arena, ML.NET is expected to start using Half<\/code>, the Apache Arrow project’s C# implementation has an open issue<\/a> for it and the DataFrame library tracks a related issue here<\/a>. As more intrinsics are unlocked in .NET for x86 and ARM processors, we expect that computation performance with Half<\/code> can be accelerated and result in more efficient code!<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
    The IEEE 754 specification defines many floating point types, including: binary16, binary32, binary64 and binary128. Most developers are familiar with binary32 (equivalent to float in C#) and binary64 (equivalent to double in C#). They provide a standard format to represent a wide range of values with a precision acceptable for many applications. .NET has always […]<\/p>\n","protected":false},"author":11778,"featured_media":58792,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[685,196,328,756,636,688,691],"tags":[4,9,30,43,96,7167],"class_list":["post-29542","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet","category-dotnet-core","category-aiml","category-csharp","category-fsharp","category-machine-learning","category-ml-dotnet","tag-net","tag-net-core","tag-announcement","tag-bcl","tag-ml-net","tag-numerics"],"acf":[],"blog_post_summary":"
    The IEEE 754 specification defines many floating point types, including: binary16, binary32, binary64 and binary128. Most developers are familiar with binary32 (equivalent to float in C#) and binary64 (equivalent to double in C#). They provide a standard format to represent a wide range of values with a precision acceptable for many applications. .NET has always […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/29542","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/11778"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=29542"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/29542\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/58792"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=29542"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=29542"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=29542"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}