{"id":32267,"date":"2023-05-29T14:42:26","date_gmt":"2023-05-29T14:42:26","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/cppblog\/?p=32267"},"modified":"2024-09-10T07:55:35","modified_gmt":"2024-09-10T07:55:35","slug":"msvc-arm64-optimizations-in-visual-studio-2022-17-6","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cppblog\/msvc-arm64-optimizations-in-visual-studio-2022-17-6\/","title":{"rendered":"MSVC ARM64 optimizations in Visual Studio 2022 17.6\u00a0"},"content":{"rendered":"

In the last couple of months, the Microsoft C++ team has been working on improving MSVC ARM64 backend performance and we are excited to have a couple of optimizations available in the <\/span>Visual Studio 2022 version 1<\/span>7.6<\/span><\/a>. These optimizations improved code-generation for both scalar ISA and SIMD ISA (NEON). Let\u2019s review some interesting optimizations in this blog.<\/span>\u00a0<\/span><\/p>\n

Before diving into technical details, we’d encourage you to create feedback here at <\/span>Developer Community<\/span><\/a> if you have found performance issues. The feedback helps us prioritize work items in our backlog. This, <\/span>optimize neon right shift into cmp<\/span><\/a>, is an example of good feedback. Including a tagged subject title, detailed description of the issue, and a simple repro simplifies our analysis work and helps us deliver a fix more quickly.<\/span>\u00a0<\/span><\/p>\n

Now, let’s see the optimizations.<\/span>\u00a0<\/span><\/p>\n

Auto-Vectorizer supports more NEON instructions with asymmetric operands<\/h2>\n

The ARM64 backend already supports some NEON instructions with asymmetric typed operands, like Add\/Subtract Long operations (SADDL\/UADDL\/SSUBL\/USUBL). These instructions add each vector element in the lower or upper half of the first source SIMD register to the corresponding vector element of the second source SIMD register and write the vector result to the destination SIMD register. The destination vector elements are twice as long as the source vector elements. Now, we have extended such support to Multiply-Add Long and Multiply-Subtract Long (SMLAL<\/code>\/UMLAL<\/code>\/SMLSL<\/code>\/UMLSL<\/code>).<\/span>\u00a0<\/span><\/p>\n

For example:<\/span>\u00a0<\/span><\/p>\n

void smlal(int * __restrict dst, int * __restrict a,\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 short * __restrict b, short * __restrict c)\u00a0\r\n{\u00a0\r\n    for (int i = 0; i < 4; i++)\u00a0\r\n        dst[i] = a[i] + b[i] * c[i];\u00a0\r\n} <\/code><\/pre>\n
In Visual Studio 2022 17.5, the code-generation was:<\/span>\u00a0<\/span><\/p>\n
sxtl\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 v19.4s,v16.4h\u00a0\r\nsxtl\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 v18.4s,v17.4h\u00a0\r\nmla\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 v20.4s,v18.4s,v19.4s\u00a0<\/code>\u202f<\/span>\u00a0<\/span><\/pre>\n
Extra signed extensions are performed on both source operands to match the type of destination. Now it has been optimized into a single <\/span>smlal v16.4s,v17.4h,v18.4h<\/code>.<\/span>\u00a0<\/span><\/p>\n
The ARM64 ISA further supports another variant for these operations, which is called Add\/Subtract Wide. For them, the <\/span>asymmetry happens between source operands, not between source and destination.<\/span>\u00a0<\/span><\/p>\n
For example:<\/span>\u00a0<\/span><\/p>\n
void saddw(int *__restrict dst, int *__restrict a, short *__restrict b)\u00a0\r\n{\u00a0\r\n    for (int i = 0; i < 4; i++)\u00a0\r\n        dst[i] = a[i] + b[i];\u00a0\r\n}<\/code><\/pre>\n
In Visual Studio 2022 17.5, the code-generation was:<\/span>\u00a0<\/span><\/p>\n
sxtl\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 v17.4s,v16.4h\u00a0\r\nadd\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 v18.4s,v17.4s,v18.4s\u00a0\u00a0<\/code><\/pre>\n
The narrow source gets extra signed extension to match the other wide source. In the 17.6 release, this has been optimized into a single <\/span>saddw v16.4s,v16.4s,v17.4h<\/code>. The same applies to UADDW<\/code>\/SSUBW<\/code>\/USUBW<\/code>.<\/span>\u00a0<\/span><\/p>\n
Auto-vectorizer now supports small types on ABS<\/code>\/MIN<\/code>\/MAX<\/code>\u00a0<\/span><\/code><\/h2>\n
ABS<\/code>\/MIN<\/code>\/MAX<\/code> have slightly complex semantics. Normally, the compiler middle-end or back-end will have a pattern matcher to recognize IR sequences with if-then-else semantics and see if they could be converted into ABS<\/code>\/MIN<\/code>\/MAX<\/code>. There is an issue when the operands are in small types (int8 or int16) though.<\/span>\u00a0<\/span><\/p>\n
\u00a0<\/span>As specified by the C++ standard, small types are promoted to int<\/code>, which is 32-bit on ARM64. This is perfect for scalar operations because they really can only operate on scalar register width. For ARM64, the smallest width is 32-bit utilizing the sub-register. However, this is not true for SIMD ISA whose minimum operation width is the width of vector lane (element). For example, ARM64 NEON supports operating on int8, int16 for a couple of operations including ABS<\/code>\/MIN<\/code>\/MAX<\/code>. So, to generate SIMD instructions operating on small element sizes and deliver higher computing throughput, the auto-vectorizer needs to do analysis and narrow the type back to the original small type when it is safe to do so.<\/span>\u00a0<\/span><\/p>\n
On top of this, ABS<\/code>\/MIN<\/code>\/MAX<\/code> bring further challenges because their source operands are scattered in different basic blocks due to their built-in if-then-else semantics. We found the MSVC auto-vectorizer may only narrow the type of the operand in one basic block making it inconsistent with operands in other basic blocks. Such inconsistency will cause the auto-vectorizer to fail matching the pattern and cancel vectorization.<\/span>\u00a0<\/span><\/p>\n
\u00a0<\/span>\u00a0For example:<\/span>\u00a0<\/span><\/p>\n
void test(signed char * __restrict a, signed char * __restrict b)\u00a0\r\n{\u00a0\r\n    for (int i = 0; i < 16; i++)\u00a0\r\n        a[i] = b[i] > 0 ? b[i] : -b[i];\u00a0\r\n}<\/code><\/pre>\n
In Visual Studio 2022 17.5, there was no vectorization, and the code-generation was:<\/span>\u00a0<\/span><\/p>\n
ldrsb\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 w8,[x1,#1]\u00a0\r\ncmp\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 w8,#0\u00a0\r\nbgt\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 |$LN16@test_abs_v|\u00a0\r\nneg\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 w8,w8\u00a0\r\nsxtb\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 w8,w8\u00a0<\/code>\u00a0<\/span>\u00a0<\/span><\/pre>\n
In the 17.6 release, the code-generation has been improved into a single <\/span>abs v16.8h,v16.8h<\/code>.<\/span><\/i>\u00a0<\/span><\/p>\n
Scalar code-generation improved based on value range analysis<\/h2>\n
When one register is compared with an immediate value, the compiler can deduce the value range of the register, and this information is useful for later optimizations, for example evaluating comparison results statically.<\/span>\u00a0<\/span><\/p>\n
MSVC already has an infrastructure for doing value range deduction, the backend just needs to teach the middle-end about the semantics of its supported comparison instructions.<\/span>\u00a0<\/span><\/p>\n
The ARM64 backend previously missed this support for CBZ<\/code> and CMP<\/code>. For <\/span>cbz reg, label<\/code>, <\/span><\/i>the <\/span>reg<\/code> must equal to zero in true path, and the same applies for CBNZ<\/code> on false path. While for <\/span>cmp reg, #imm<\/code>, <\/span><\/i>the <\/span>reg<\/code> must equal<\/span> imm<\/code> in true path. Knowing such equivalence, the compiler could simplify code-generation. Let\u2019s see a simple example:<\/span>\u00a0<\/span><\/p>\n
int cal(int a)\u00a0\r\n{\u00a0\r\n    if (!a)\u00a0\r\n\u00a0\u00a0\u00a0\u00a0    return a;\u00a0\r\n\u00a0\u00a0\u00a0\u00a0else\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0return a * 2;\u00a0\r\n}<\/code>\u00a0<\/span><\/pre>\n
In Visual Studio 2022 17.5, MSVC was generating the following instruction sequence:<\/span>\u00a0<\/span><\/p>\n
|int cal(int)| PROC\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 cbnz\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 w0,|$LN2@cal|\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 mov\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 w0,#0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 <- redundant mov\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ret\u00a0\r\n|$LN2@cal|\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 lsl\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 w0,w0,#1\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ret\u00a0<\/code><\/pre>\n
The <\/span>mov w0, #0<\/code> is redundant because when the execution reaches there, <\/span>w0<\/code> must be zero. After our recent optimization, the code-generation is optimal in release 17.6:<\/span>\u00a0<\/span><\/p>\n
|int cal(int)| PROC\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 lsl\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 w8,w0,#1\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 cmp\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 w0,#0\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 cseleq\u00a0\u00a0\u00a0\u00a0\u00a0 w0,wzr,w8\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ret <\/code><\/pre>\n
Scalar code-generation improved to catch more if-conversion opportunities<\/h2>\n
You may have noticed there is another difference in the code-generation for the above test case. CSELEQ<\/code> is used instead of branch. The ARM64 ISA supports the CSEL<\/code> instruction to do if-conversion which saves both execution cycles and code size. Previously, MSVC couldn\u2019t generate CSEL<\/code> when the selected value came from a return statement. We fixed this in the 17.6 release, so for the above example CSELEQ<\/code> is used instead of CBNZ<\/code>. <\/span>\u00a0<\/span><\/p>\n
In summary, for the if-return-else-return pattern, the ARM64 backend has been taught to generate a CSEL<\/code> instruction if the return statement is in any of the following operations:<\/span>\u00a0<\/span><\/p>\n