We hear that many of you would like to see more details of the improvements which the MSVC backend team have been working on recently. This blog post presents some of the optimizations the team has implemented for Visual Studio 2022. It was co-written by one of our backend team leads, Eric Brumer, and our developer advocate, Sy Brand. Keep an eye out for more posts in the future which will dig into other optimizations!<\/p>\n
Changing the endianness<\/a> of an integer can be an important operation in contexts where data is being transmitted between processors with different byte orders, or over the network. This is often referred to as a “byteswap”. C++23 will add a In Visual Studio 2019 16.11, this generated the following code for x64, and a similarly long sequence of instructions for Arm64:<\/p>\n x64 and Arm64 both have instructions which carry out a byteswap, which should ideally be used instead. Visual Studio 2022 17.3 introduced automatic byteswap identification, and now outputs the following on x64 with On Arm64, the results are much the same:<\/p>\n This optimization isn’t just doing a simple pattern match on your code, it’s a generalized bit tracker. For example, this bit order reverser gets optimized to a single rbit<\/span><\/span>\u00a0on Arm64:<\/p>\n For sake of code simplicity, we may choose to write a branch inside a loop whose condition is invariable in each iteration. For example, if we want to pet all our cats, and if it’s feeding time, we also want to feed them, we could write this:<\/p>\n Since As of Visual Studio 2022 version 17.1, MSVC may carry out loop unswitching at Here is the x64 generated by Visual Studio 2019 version 16.11 with The key part to note here are the lines directly under the Here is the code generated now with The code is now longer because two versions of the loop body are now generated, under the We have improved optimization of chains of Say we have three blankets and we want to give one to our cat. The one we pick shouldn’t be too hard. It shouldn’t be too soft. It should be just right.<\/p>\n We could write a function to give us the Just Right blanket from three, by picking the middle one. It could look something like this:<\/p>\n In VS2019, this code was generated for x64 with Arm64 codegen is similarly inefficient. Both x64 and Arm64 have single instructions for scalar floating point min and max, which we now use in VS2022 at And for Arm64:<\/p>\n Say I run a cat shelter with 32 cats and want to count how many crunchies they leave behind in their bowls after mealtime. So I write a function which takes a pointer to the first bowl, and sum it like so (yes, I know I could use This all works and generates good code! But then I realize that my desk is actually at the far end of the room, so I need to walk all the way to the start of the line to begin counting. I decide to instead take a pointer to the last<\/em> bowl and work backwards:<\/p>\n Unfortunately, if I was using VS2019, this loop would not be vectorized<\/a>. Here is the code generated with The loop is unrolled<\/a> but it is not vectorized.<\/p>\n We enabled vectorization for backwards-strided loops in VS2022 17.1. The code generated will depend a lot on the flags you use, particularly the Here is the code generated for This both unrolls and vectorizes the loop. Fully explaining AVX2 vector instructions is a job for a different blog post (maybe check out this one<\/a>), but the basic idea is that all those We hope you found these details interesting! If you have ideas for similar posts you’d like to see, please let us know. We are also interested in your feedback to continue to improve our tools. The comments below are open. Feedback can also be shared through Developer Community<\/a>. You can also reach us on Twitter (@VisualC<\/a>), or via email at\u00a0visualcpp@microsoft.com<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":" This blog post presents some of the optimizations the backend team has implemented for Visual Studio 2022.<\/p>\n","protected":false},"author":706,"featured_media":35994,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[3946,1,218],"tags":[],"class_list":["post-31340","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-backend","category-cplusplus","category-performance"],"acf":[],"blog_post_summary":" This blog post presents some of the optimizations the backend team has implemented for Visual Studio 2022.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts\/31340","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/users\/706"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/comments?post=31340"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts\/31340\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/media\/35994"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/media?parent=31340"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/categories?post=31340"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/tags?post=31340"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}std::byteswap<\/code><\/a> function to the standard library (already implemented in VS2022 17.1), but for codebases today it’s common to see custom implementations, which might look something like this:<\/p>\nint byteswap(int n) {\r\n return ((n & 0xff) << 24u) |\r\n ((n & 0xff00) << 8u) |\r\n ((n & 0xff0000) >> 8u) |\r\n ((n & 0xff000000) >> 24u);\r\n}<\/code><\/pre>\n mov eax, ecx\r\n mov edx, ecx\r\n and eax, 65280\r\n shl edx, 16\r\n or eax, edx\r\n mov edx, ecx\r\n shl eax, 8\r\n sar edx, 8\r\n and edx, 65280\r\n shr ecx, 24\r\n or eax, edx\r\n or eax, ecx\r\n ret 0<\/code><\/pre>\n\/O2<\/code>:<\/p>\n bswap ecx\r\n mov eax, ecx\r\n ret 0<\/code><\/pre>\n rev w0,w0\r\n ret<\/code><\/pre>\nunsigned __int64 bitswap(unsigned __int64 x) {\r\n x = _byteswap_uint64(x);\r\n x = (x & 0xaaaaaaaaaaaaaaaa) >> 1 | (x & 0x5555555555555555) << 1;\r\n x = (x & 0xcccccccccccccccc) >> 2 | (x & 0x3333333333333333) << 2;\r\n x = (x & 0xf0f0f0f0f0f0f0f0) >> 4 | (x & 0x0f0f0f0f0f0f0f0f) << 4;\r\n return x;\r\n}<\/code><\/pre>\nLoop Unswitching<\/h2>\n
void process_cats(std::span<cat> cats, bool is_feeding_time) {\r\n for (auto&& c : cats) {\r\n if (is_feeding_time) {\r\n feed(c);\r\n }\r\n pet(c);\r\n }\r\n}<\/code><\/pre>\nis_feeding_time<\/code> never changes inside the function and has no side-effects, it may be wasteful to carry out that check every iteration. Better hardware branch predictors<\/a> will minimize this impact, but in many cases we can see performance wins by carrying out loop unswitching<\/a>. This hoists the branch outside of the loop, generating code similar to this C++:<\/p>\nvoid process_cats(std::span<cat> cats, bool is_feeding_time) {\r\n if (is_feeding_time) {\r\n for (auto&& c : cats) {\r\n feed(c);\r\n pet(c);\r\n }\r\n }\r\n else {\r\n for (auto&& c : cats) {\r\n pet(c);\r\n }\r\n }\r\n}<\/code><\/pre>\n\/O2<\/code>. This is a heuristic<\/a>-driven optimization which may or may not be carried out depending on analysis results. This is because loop unswitching duplicates the loop body, which may affect instruction cache performance. You may pass the \/Qvec-report:1<\/code> flag to see a report on which loops were unswitched.<\/p>\n\/O2<\/code>:<\/p>\n$LN15:\r\n mov QWORD PTR [rsp+8], rbx\r\n mov QWORD PTR [rsp+16], rsi\r\n push rdi\r\n sub rsp, 32 \r\n mov rbx, QWORD PTR [rcx]\r\n movzx esi, dl\r\n mov rax, QWORD PTR [rcx+8]\r\n lea rdi, QWORD PTR [rbx+rax*4]\r\n cmp rbx, rdi\r\n je SHORT $LN3@process_ca\r\n$LL4@process_ca:\r\n test sil, sil\r\n je SHORT $LN5@process_ca\r\n mov ecx, DWORD PTR [rbx]\r\n call void feed(cat) \r\n$LN5@process_ca:\r\n mov ecx, DWORD PTR [rbx]\r\n call void pet(cat) \r\n add rbx, 4\r\n cmp rbx, rdi\r\n jne SHORT $LL4@process_ca\r\n$LN3@process_ca:\r\n mov rbx, QWORD PTR [rsp+48]\r\n mov rsi, QWORD PTR [rsp+56]\r\n add rsp, 32 \r\n pop rdi\r\n ret 0<\/code><\/pre>\n$LL4@process_ca<\/code> label, which carry out the test on is_feeding_time<\/code> and branch on the result, all inside the loop.<\/p>\n\/O2<\/code>:<\/p>\n$LN22:\r\n mov QWORD PTR [rsp+8], rbx\r\n push rdi\r\n sub rsp, 32 \r\n mov rbx, QWORD PTR [rcx]\r\n mov rax, QWORD PTR [rcx+8]\r\n lea rdi, QWORD PTR [rbx+rax*4]\r\n cmp rbx, rdi\r\n je SHORT $LN14@process_ca\r\n test dl, dl\r\n je SHORT $LL11@process_ca\r\n npad 2\r\n$LL4@process_ca:\r\n mov ecx, DWORD PTR [rbx]\r\n call void feed(cat)\r\n mov ecx, DWORD PTR [rbx]\r\n call void pet(cat) \r\n add rbx, 4\r\n cmp rbx, rdi\r\n jne SHORT $LL4@process_ca\r\n mov rbx, QWORD PTR [rsp+48]\r\n add rsp, 32 \r\n pop rdi\r\n ret 0\r\n$LL11@process_ca:\r\n mov ecx, DWORD PTR [rbx]\r\n call void pet(cat)\r\n add rbx, 4\r\n cmp rbx, rdi\r\n jne SHORT $LL11@process_ca\r\n$LN14@process_ca:\r\n mov rbx, QWORD PTR [rsp+48]\r\n add rsp, 32 \r\n pop rdi\r\n ret 0<\/code><\/pre>\n$LL4@process_ca<\/code> and $LL11@process_ca<\/code> labels. But also note that the branch occurs in the entry block of the function and selects between the two loop body versions:<\/p>\n cmp rbx, rdi\r\n je SHORT $LN14@process_ca\r\n test dl, dl\r\n je SHORT $LL11@process_ca<\/code><\/pre>\nMin\/Max Chains<\/h2>\n
std::min<\/code> and std::max<\/code> as of Visual Studio 2022 version 17.0.<\/p>\nusing softness = float;\r\nsoftness just_right_blanket(softness a, softness b, softness c) {\r\n return std::max(std::min(a,b), std::min(std::max(a,b),c));\r\n}<\/code><\/pre>\n\/O2 \/fp:fast<\/code>:<\/p>\n comiss xmm1, xmm0\r\n lea rcx, QWORD PTR a$[rsp]\r\n lea rax, QWORD PTR b$[rsp]\r\n lea rdx, QWORD PTR a$[rsp]\r\n movss DWORD PTR [rsp+16], xmm1\r\n movss DWORD PTR [rsp+8], xmm0\r\n cmovbe rax, rcx\r\n movss DWORD PTR [rsp+24], xmm2\r\n lea rcx, QWORD PTR c$[rsp]\r\n comiss xmm2, DWORD PTR [rax]\r\n cmovae rcx, rax\r\n lea rax, QWORD PTR b$[rsp]\r\n comiss xmm0, xmm1\r\n cmovbe rax, rdx\r\n movss xmm1, DWORD PTR [rax]\r\n comiss xmm1, DWORD PTR [rcx]\r\n cmovb rax, rcx\r\n movss xmm0, DWORD PTR [rax]\r\n ret 0<\/code><\/pre>\n\/O2<\/code> and above with \/fp:fast<\/code>. Here is the x64 code now:<\/p>\n movaps xmm3, xmm0\r\n maxss xmm0, xmm1\r\n minss xmm3, xmm1\r\n minss xmm0, xmm2\r\n maxss xmm0, xmm3\r\n ret 0<\/code><\/pre>\n fmax s16,s0,s1\r\n fmin s17,s0,s1\r\n fmin s18,s16,s2\r\n fmax s0,s18,s17\r\n ret<\/code><\/pre>\nBackwards Loop Vectorization<\/h2>\n
std::accumulate<\/code>):<\/p>\nint count_leftovers(int* bowl_ptr) {\r\n int result = 0;\r\n\r\n for (int i = 0; i < 32; ++i, ++bowl_ptr) {\r\n result += *bowl_ptr;\r\n }\r\n return result;\r\n}<\/code><\/pre>\nint count_leftovers(int* bowl_ptr) {\r\n int result = 0;\r\n\r\n \/\/ change ++bowl_ptr to --bowl_ptr\r\n for (int i = 0; i < 32; ++i, --bowl_ptr) {\r\n result += *bowl_ptr;\r\n }\r\n return result;\r\n}<\/code><\/pre>\n\/O2<\/code>:<\/p>\n xor eax, eax\r\n mov edx, eax\r\n mov r8d, eax\r\n mov r9d, eax\r\n add rcx, -8\r\n lea r10d, QWORD PTR [rax+8]\r\n$LL4@count_left:\r\n add eax, DWORD PTR [rcx+8]\r\n add r9d, DWORD PTR [rcx+4]\r\n add r8d, DWORD PTR [rcx]\r\n add edx, DWORD PTR [rcx-4]\r\n lea rcx, QWORD PTR [rcx-16]\r\n sub r10, 1\r\n jne SHORT $LL4@count_left\r\n lea ecx, DWORD PTR [rdx+r8]\r\n add ecx, r9d\r\n add eax, ecx\r\n ret 0<\/code><\/pre>\n\/arch<\/code><\/a> flag for enabling use of AVX<\/a> instructions instead of the default SSE2<\/a> ones.<\/p>\n\/O2 \/arch:AVX2<\/code>:<\/p>\n vpxor xmm2, xmm2, xmm2\r\n vpxor xmm3, xmm3, xmm3\r\n mov eax, 2\r\n npad 3\r\n$LL4@count_left:\r\n vmovd xmm1, DWORD PTR [rcx]\r\n vpinsrd xmm1, xmm1, DWORD PTR [rcx-4], 1\r\n vpinsrd xmm1, xmm1, DWORD PTR [rcx-8], 2\r\n vpinsrd xmm1, xmm1, DWORD PTR [rcx-12], 3\r\n vmovd xmm0, DWORD PTR [rcx-16]\r\n vpinsrd xmm0, xmm0, DWORD PTR [rcx-20], 1\r\n vpinsrd xmm0, xmm0, DWORD PTR [rcx-24], 2\r\n vpinsrd xmm0, xmm0, DWORD PTR [rcx-28], 3\r\n lea rcx, QWORD PTR [rcx-64]\r\n vinsertf128 ymm0, ymm1, xmm0, 1\r\n vmovd xmm1, DWORD PTR [rcx+32]\r\n vpinsrd xmm1, xmm1, DWORD PTR [rcx+28], 1\r\n vpinsrd xmm1, xmm1, DWORD PTR [rcx+24], 2\r\n vpinsrd xmm1, xmm1, DWORD PTR [rcx+20], 3\r\n vpaddd ymm2, ymm0, ymm2\r\n vmovd xmm0, DWORD PTR [rcx+16]\r\n vpinsrd xmm0, xmm0, DWORD PTR [rcx+12], 1\r\n vpinsrd xmm0, xmm0, DWORD PTR [rcx+8], 2\r\n vpinsrd xmm0, xmm0, DWORD PTR [rcx+4], 3\r\n vinsertf128 ymm0, ymm1, xmm0, 1\r\n vpaddd ymm3, ymm0, ymm3\r\n sub rax, 1\r\n jne $LL4@count_left\r\n vpaddd ymm0, ymm3, ymm2\r\n vphaddd ymm1, ymm0, ymm0\r\n vphaddd ymm2, ymm1, ymm1\r\n vextracti128 xmm0, ymm2, 1\r\n vpaddd xmm0, xmm2, xmm0\r\n vmovd eax, xmm0\r\n vzeroupper\r\n ret 0<\/code><\/pre>\nvpinsrd<\/code><\/a> instructions are loading the data from memory into vector registers, then the vpaddd<\/code><\/a>\/vphaddd<\/code><\/a> instructions carry out addition on big chunks of data all at the same time.<\/p>\nSend us your feedback<\/h2>\n