{"id":57952,"date":"2025-09-10T06:00:00","date_gmt":"2025-09-10T13:00:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/dotnet\/?p=57952"},"modified":"2025-11-10T15:19:27","modified_gmt":"2025-11-10T23:19:27","slug":"performance-improvements-in-net-10","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/performance-improvements-in-net-10\/","title":{"rendered":"Performance Improvements in .NET 10"},"content":{"rendered":"

My kids love<\/em> “Frozen”. They can sing every word, re-enact every scene, and provide detailed notes on the proper sparkle of Elsa’s ice dress. I’ve seen the movie more times than I can recount, to the point where, if you’ve seen me do any live coding, you’ve probably seen my subconscious incorporate an Arendelle reference or two. After so many viewings, I began paying closer attention to the details, like how at the very beginning of the film the ice harvesters are singing a song that subtly foreshadows the story’s central conflicts, the characters’ journeys, and even the key to resolving the climax. I’m slightly ashamed to admit I didn’t comprehend this connection until viewing number ten or so, at which point I also realized I had no idea if this ice harvesting was actually “a thing” or if it was just a clever vehicle for Disney to spin a yarn. Turns out, as I subsequently researched, it’s quite real.<\/p>\n

In the 19th century, before refrigeration, ice was an incredibly valuable commodity. Winters in the northern United States turned ponds and lakes into seasonal gold mines. The most successful operations ran with precision: workers cleared snow from the surface so the ice would grow thicker and stronger, and they scored the surface into perfect rectangles using horse-drawn plows, turning the lake into a frozen checkerboard. Once the grid was cut, teams with long saws worked to free uniform blocks weighing several hundred pounds each. These blocks were floated along channels of open water toward the shore, at which point men with poles levered the blocks up ramps and hauled them into storage. Basically, what the movie shows.<\/p>\n

The storage itself was an art. Massive wooden ice houses, sometimes holding tens of thousands of tons, were lined with insulation, typically straw. Done well, this insulation could keep the ice solid for months, even through summer heat. Done poorly, you would open the doors to slush. And for those moving ice over long distances, typically by ship, every degree, every crack in the insulation, every extra day in transit meant more melting and more loss.<\/p>\n

Enter Frederic Tudor, the “Ice King” of Boston. He was obsessed with systemic efficiency. Where competitors saw unavoidable loss, Tudor saw a solvable problem. After experimenting with different insulators, he leaned on cheap sawdust, a lumber mill byproduct that outperformed straw, packing it densely around the ice to cut melt losses significantly. For harvesting efficiency, his operations adopted Nathaniel Jarvis Wyeth’s grid-scoring system, which produced uniform blocks that could be packed tightly, minimizing air gaps that would otherwise increase exposure in a ship’s hold. And to shorten the critical time between shore and ship, Tudor built out port infrastructure and depots near docks, allowing ships to load and unload much faster. Each change, from tools to ice house design to logistics, amplified the last, turning a risky local harvest into a reliable global trade. With Tudor’s enhancements, he had solid ice arriving in places like Havana, Rio de Janeiro, and even Calcutta (a voyage of four months in the 1830s). His performance gains allowed the product to survive journeys that were previously unthinkable.<\/p>\n

What made Tudor’s ice last halfway around the world wasn’t one big idea. It was a plethora of small improvements, each multiplying the effect of the last. In software development, the same principle holds: big leaps forward in performance rarely come from a single sweeping change, rather from hundreds or thousands of targeted optimizations that compound into something transformative. .NET 10’s performance story isn’t about one Disney-esque magical idea; it’s about carefully shaving off nanoseconds here and tens of bytes there, streamlining operations that are executed trillions of times.<\/p>\n

In the rest of this post, just as we did in Performance Improvements in .NET 9<\/a>, .NET 8<\/a>, .NET 7<\/a>, .NET 6<\/a>, .NET 5<\/a>, .NET Core 3.0<\/a>, .NET Core 2.1<\/a>, and .NET Core 2.0<\/a>, we’ll dig into hundreds of the small but meaningful and compounding performance improvements since .NET 9 that make up .NET 10’s story (if you instead stay on LTS releases and thus are upgrading from .NET 8 instead of from .NET 9, you’ll see even more improvements based on the aggregation from all the improvements in .NET 9<\/a> as well). So, without further ado, go grab a cup of your favorite hot beverage (or, given my intro, maybe something a bit more frosty), sit back, relax, and “Let It Go”!<\/p>\n

Or, hmm, maybe, let’s push performance “Into the Unknown”?<\/p>\n

Let .NET 10 performance “Show Yourself”?<\/p>\n

“Do You Want To Build a Snowman<\/del> Fast Service?”<\/p>\n

I’ll see myself out.<\/p>\n

Benchmarking Setup<\/h2>\n

As in previous posts, this tour is chock full of micro-benchmarks intended to showcase various performance improvements. Most of these benchmarks are implemented using BenchmarkDotNet 0.15.2<\/a>, with a simple setup for each.<\/p>\n

To follow along, make sure you have .NET 9<\/a> and .NET 10<\/a> installed, as most of the benchmarks compare the same test running on each. Then, create a new C# project in a new benchmarks<\/code> directory:<\/p>\n

dotnet new console -o benchmarks\r\ncd benchmarks<\/code><\/pre>\n
That will produce two files in the benchmarks<\/code> directory: benchmarks.csproj<\/code>, which is the project file with information about how the application should be compiled, and Program.cs<\/code>, which contains the code for the application. Finally, replace everything in benchmarks.csproj<\/code> with this:<\/p>\n
<Project Sdk=\"Microsoft.NET.Sdk\">\r\n\r\n  <PropertyGroup>\r\n    <OutputType>Exe<\/OutputType>\r\n    <TargetFrameworks>net10.0;net9.0<\/TargetFrameworks>\r\n    <LangVersion>Preview<\/LangVersion>\r\n    <ImplicitUsings>enable<\/ImplicitUsings>\r\n    <Nullable>enable<\/Nullable>\r\n    <ServerGarbageCollection>true<\/ServerGarbageCollection>\r\n  <\/PropertyGroup>\r\n\r\n  <ItemGroup>\r\n    <PackageReference Include=\"BenchmarkDotNet\" Version=\"0.15.2\" \/>\r\n  <\/ItemGroup>\r\n\r\n<\/Project><\/code><\/pre>\n
With that, we’re good to go. Unless otherwise noted, I’ve tried to make each benchmark standalone; just copy\/paste its whole contents into the Program.cs file, overwriting everything that’s there, and then run the benchmarks. Each test includes at its top a comment for the dotnet<\/code> command to use to run the benchmark. It’s typically something like this:<\/p>\n
dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0<\/code><\/pre>\n
which will run the benchmark in release on both .NET 9 and .NET 10 and show the compared results. The other common variation, used when the benchmark should only be run on .NET 10 (typically because it’s comparing two approaches rather than comparing one thing on two versions), is the following:<\/p>\n
dotnet run -c Release -f net10.0 --filter \"*\"<\/code><\/pre>\n
Throughout the post, I’ve shown many benchmarks and the results I received from running them. Unless otherwise stated (e.g. because I’m demonstrating an OS-specific improvement), the results shown are from running them on Linux (Ubuntu 24.04.1) on an x64 processor.<\/p>\n
BenchmarkDotNet v0.15.2, Linux Ubuntu 24.04.1 LTS (Noble Numbat)\r\n11th Gen Intel Core i9-11950H 2.60GHz, 1 CPU, 16 logical and 8 physical cores\r\n.NET SDK 10.0.100-rc.1.25451.107\r\n  [Host]     : .NET 9.0.9 (9.0.925.41916), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI<\/code><\/pre>\n
As always, a quick disclaimer: these are micro-benchmarks, timing operations so short you’d miss them by blinking (but when such operations run millions of times, the savings really add up). The exact numbers you get will depend on your hardware, your operating system, what else your machine is juggling at the moment, how much coffee you’ve had since breakfast, and perhaps whether Mercury is in retrograde. In other words, don’t expect your results to match mine exactly, but I’ve picked tests that should still be reasonably reproducible in the real world.<\/p>\n
Now, let’s start at the bottom of the stack. Code generation.<\/p>\n
JIT<\/h2>\n
Among all areas of .NET, the Just-In-Time (JIT) compiler stands out as one of the most impactful. Every .NET application, whether a small console tool or a large-scale enterprise service, ultimately relies on the JIT to turn intermediate language (IL) code into optimized machine code. Any enhancement to the JIT’s generated code quality has a ripple effect, improving performance across the entire ecosystem without requiring developers to change any of their own code or even recompile their C#. And with .NET 10, there\u2019s no shortage of these improvements.<\/p>\n
Deabstraction<\/h3>\n
As with many languages, .NET historically has had an “abstraction penalty,” those extra allocations and indirections that can occur when using high-level language features like interfaces, iterators, and delegates. Each year, the JIT gets better and better at optimizing away layers of abstraction, so that developers get to write simple code and still get great performance. .NET 10 continues this tradition. The result is that idiomatic C# (using interfaces, foreach<\/code> loops, lambdas, etc.) runs even closer to the raw speed of meticulously crafted and hand-tuned code.<\/p>\n
Object Stack Allocation<\/h4>\n
One of the most exciting areas of deabstraction progress in .NET 10 is the expanded use of escape analysis to enable stack allocation of objects. Escape analysis is a compiler technique to determine whether an object allocated in a method escapes that method, meaning determining whether that object is reachable after the method returns (for example, by being stored in a field or returned to the caller) or used in some way that the runtime can’t track within the method (like passed to an unknown callee). If the compiler can prove an object doesn’t escape, then that object’s lifetime is bounded by the method, and it can be allocated on the stack instead of on the heap. Stack allocation is much cheaper (just pointer bumping for allocation and automatic freeing when the method exits) and reduces GC pressure because, well, the object doesn’t need to be tracked by the GC. .NET 9 had already introduced some limited escape analysis and stack allocation support; .NET 10 takes this significantly further.<\/p>\n
dotnet\/runtime#115172<\/a> teaches the JIT how to perform escape analysis related to delegates, and in particular that a delegate’s Invoke<\/code> method (which is implemented by the runtime) does not stash away the this<\/code> reference. Then if escape analysis can prove that the delegate’s object reference is something that otherwise hasn’t escaped, the delegate can effectively evaporate. Consider this benchmark:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[MemoryDiagnoser(displayGenColumns: false)]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\", \"y\")]\r\npublic partial class Tests\r\n{\r\n    [Benchmark]\r\n    [Arguments(42)]\r\n    public int Sum(int y)\r\n    {\r\n        Func<int, int> addY = x => x + y;\r\n        return DoubleResult(addY, y);\r\n    }\r\n\r\n    private int DoubleResult(Func<int, int> func, int arg)\r\n    {\r\n        int result = func(arg);\r\n        return result + result;\r\n    }\r\n}<\/code><\/pre>\n
If we just run this benchmark and compare .NET 9 and .NET 10, we can immediately tell something interesting is happening.<\/p>\n\n\n\n\n\n\n
Method<\/th>\nRuntime<\/th>\nMean<\/th>\nRatio<\/th>\nCode Size<\/th>\nAllocated<\/th>\nAlloc Ratio<\/th>\n<\/tr>\n<\/thead>\n
Sum<\/td>\n.NET 9.0<\/td>\n19.530 ns<\/td>\n1.00<\/td>\n118 B<\/td>\n88 B<\/td>\n1.00<\/td>\n<\/tr>\n
Sum<\/td>\n.NET 10.0<\/td>\n6.685 ns<\/td>\n0.34<\/td>\n32 B<\/td>\n24 B<\/td>\n0.27<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
The C# code for Sum<\/code> belies complicated code generation by the C# compiler. It needs to create a Func<int, int><\/code>, which is “closing over” the y<\/code> “local”. That means the compiler needs to “lift” y<\/code> to no longer be an actual local, and instead live as a field on an object; the delegate can then point to a method on that object, giving it access to y<\/code>. This is approximately what the IL generated by the C# compiler looks like when decompiled to C#:<\/p>\n
public int Sum(int y)\r\n{\r\n    <>c__DisplayClass0_0 c = new();\r\n    c.y = y;\r\n\r\n    Func<int, int> func = new(c.<Sum>b__0);\r\n\r\n    return DoubleResult(func, c.y);\r\n}\r\n\r\nprivate sealed class <>c__DisplayClass0_0\r\n{\r\n    public int y;\r\n\r\n    internal int <Sum>b__0(int x) => x + y;\r\n}<\/code><\/pre>\n
From that, we can see the closure is resulting in two allocations: an allocation for the “display class” (what the C# compiler calls these closure types) and an allocation for the delegate that points to the <Sum>b__0<\/code> method on that display class instance. That’s what’s accounting for the 88<\/code> bytes of allocation in the .NET 9 results: the display class is 24 bytes, and the delegate is 64 bytes. In the .NET 10 version, though, we only see a 24 byte allocation; that’s because the JIT has successfully elided the delegate allocation. Here is the resulting assembly code:<\/p>\n
; .NET 9\r\n; Tests.Sum(Int32)\r\n       push      rbp\r\n       push      r15\r\n       push      rbx\r\n       lea       rbp,[rsp+10]\r\n       mov       ebx,esi\r\n       mov       rdi,offset MT_Tests+<>c__DisplayClass0_0\r\n       call      CORINFO_HELP_NEWSFAST\r\n       mov       r15,rax\r\n       mov       [r15+8],ebx\r\n       mov       rdi,offset MT_System.Func<System.Int32, System.Int32>\r\n       call      CORINFO_HELP_NEWSFAST\r\n       mov       rbx,rax\r\n       lea       rdi,[rbx+8]\r\n       mov       rsi,r15\r\n       call      CORINFO_HELP_ASSIGN_REF\r\n       mov       rax,offset Tests+<>c__DisplayClass0_0.<Sum>b__0(Int32)\r\n       mov       [rbx+18],rax\r\n       mov       esi,[r15+8]\r\n       cmp       [rbx+18],rax\r\n       jne       short M00_L01\r\n       mov       rax,[rbx+8]\r\n       add       esi,[rax+8]\r\n       mov       eax,esi\r\nM00_L00:\r\n       add       eax,eax\r\n       pop       rbx\r\n       pop       r15\r\n       pop       rbp\r\n       ret\r\nM00_L01:\r\n       mov       rdi,[rbx+8]\r\n       call      qword ptr [rbx+18]\r\n       jmp       short M00_L00\r\n; Total bytes of code 112\r\n\r\n; .NET 10\r\n; Tests.Sum(Int32)\r\n       push      rbx\r\n       mov       ebx,esi\r\n       mov       rdi,offset MT_Tests+<>c__DisplayClass0_0\r\n       call      CORINFO_HELP_NEWSFAST\r\n       mov       [rax+8],ebx\r\n       mov       eax,[rax+8]\r\n       mov       ecx,eax\r\n       add       eax,ecx\r\n       add       eax,eax\r\n       pop       rbx\r\n       ret\r\n; Total bytes of code 32<\/code><\/pre>\n
In both .NET 9 and .NET 10, the JIT successfully inlined DoubleResult<\/code>, such that the delegate doesn’t escape, but then in .NET 10, it’s able to stack allocate it. Woo hoo! There’s obviously still future opportunity here, as the JIT doesn’t elide the allocation of the closure object, but that should be addressable with some more effort, hopefully in the near future.<\/p>\n
dotnet\/runtime#104906<\/a> from @hez2010<\/a> and dotnet\/runtime#112250<\/a> extend this kind of analysis and stack allocation to arrays. How many times have you written code like this?<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\nusing System.Runtime.CompilerServices;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[MemoryDiagnoser(displayGenColumns: false)]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    [Benchmark]\r\n    public void Test()\r\n    {\r\n        Process(new string[] { \"a\", \"b\", \"c\" });\r\n\r\n        static void Process(string[] inputs)\r\n        {\r\n            foreach (string input in inputs)\r\n            {\r\n                Use(input);\r\n            }\r\n\r\n            [MethodImpl(MethodImplOptions.NoInlining)]\r\n            static void Use(string input) { }\r\n        }\r\n    }\r\n}<\/code><\/pre>\n
Some method I want to call accepts an array of inputs and does something for each input. I need to allocate an array to pass my inputs in, either explicitly, or maybe implicitly due to using params<\/code> or a collection expression. Ideally moving forward there would be an overload of such a Process<\/code> method that accepted a ReadOnlySpan<string><\/code> instead of a string[]<\/code>, and I could then avoid the allocation by construction. But for all of these cases where I’m forced to create an array, .NET 10 comes to the rescue.<\/p>\n\n\n\n\n\n\n
Method<\/th>\nRuntime<\/th>\nMean<\/th>\nRatio<\/th>\nAllocated<\/th>\nAlloc Ratio<\/th>\n<\/tr>\n<\/thead>\n
Test<\/td>\n.NET 9.0<\/td>\n11.580 ns<\/td>\n1.00<\/td>\n48 B<\/td>\n1.00<\/td>\n<\/tr>\n
Test<\/td>\n.NET 10.0<\/td>\n3.960 ns<\/td>\n0.34<\/td>\n–<\/td>\n0.00<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
The JIT was able to inline Process<\/code>, see that the array never leaves the frame, and stack allocate it.<\/p>\n
Of course, now that we’re able to stack allocate arrays, we also want to be able to deal with a common way those arrays are used: via spans. dotnet\/runtime#113977<\/a> and dotnet\/runtime#116124<\/a> teach escape analysis to be able to reason about the fields in structs, which includes Span<T><\/code>, as it’s “just” a struct that stores a ref T<\/code> field and an int<\/code> length field.<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\nusing System.Runtime.CompilerServices;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[MemoryDiagnoser(displayGenColumns: false)]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private byte[] _buffer = new byte[3];\r\n\r\n    [Benchmark]\r\n    public void Test() => Copy3Bytes(0x12345678, _buffer);\r\n\r\n    [MethodImpl(MethodImplOptions.NoInlining)]\r\n    private static void Copy3Bytes(int value, Span<byte> dest) =>\r\n        BitConverter.GetBytes(value).AsSpan(0, 3).CopyTo(dest);\r\n}<\/code><\/pre>\n
Here, we’re using BitConverter.GetBytes<\/code>, which allocates a byte[]<\/code> containing the bytes from the input (in this case, it’ll be a four-byte array for the int<\/code>), then we slice off three of the four bytes, and we copy them to the destination span.<\/p>\n\n\n\n\n\n\n
Method<\/th>\nRuntime<\/th>\nMean<\/th>\nRatio<\/th>\nAllocated<\/th>\nAlloc Ratio<\/th>\n<\/tr>\n<\/thead>\n
Test<\/td>\n.NET 9.0<\/td>\n9.7717 ns<\/td>\n1.04<\/td>\n32 B<\/td>\n1.00<\/td>\n<\/tr>\n
Test<\/td>\n.NET 10.0<\/td>\n0.8718 ns<\/td>\n0.09<\/td>\n–<\/td>\n0.00<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
In .NET 9, we get the 32-byte allocation we’d expect for the byte[]<\/code> in GetBytes<\/code> (every object on 64-bit is at least 24 bytes, which will include the four bytes for the array’s length, and then the four bytes for the data will be in slots 24-27, and the size will be padded up to the next word boundary, for 32). In .NET 10, with GetBytes<\/code> and AsSpan<\/code> inlined, the JIT can see that the array doesn’t escape, and a stack allocated version of it can be used to seed the span, just as if it were created from any other stack allocation (like stackalloc<\/code>). (This case also needed a little help from dotnet\/runtime#113093<\/a>, which taught the JIT that certain span operations, like the Memmove<\/code> used internally by CopyTo<\/code>, are non-escaping.)<\/p>\n
Devirtualization<\/h4>\n
Interfaces and virtual methods are a critical aspect of .NET and the abstractions it enables. Being able to unwind these abstractions and “devirtualize” is then an important job for the JIT, which has taken notable leaps in capabilities here in .NET 10.<\/p>\n
While arrays are one of the most central features provided by C# and .NET, and while the JIT exerts a lot of energy and does a great job optimizing many aspects of arrays, one area in particular has caused it pain: an array’s interface implementations. The runtime manufactures a bunch of interface implementations for T[]<\/code>, and because they’re implemented differently from literally every other interface implementation in .NET, the JIT hasn’t been able to apply the same devirtualization capabilities it’s applied elsewhere. And, for anyone who’s dived deep into micro-benchmarks, this can lead to some odd observations. Here’s a performance comparison between iterating over a ReadOnlyCollection<T><\/code> using a foreach<\/code> loop (going through its enumerator) and using a for<\/code> loop (indexing on each element).<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\"\r\n\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\nusing System.Collections.ObjectModel;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private ReadOnlyCollection<int> _list = new(Enumerable.Range(1, 1000).ToArray());\r\n\r\n    [Benchmark]\r\n    public int SumEnumerable()\r\n    {\r\n        int sum = 0;\r\n        foreach (var item in _list)\r\n        {\r\n            sum += item;\r\n        }\r\n        return sum;\r\n    }\r\n\r\n    [Benchmark]\r\n    public int SumForLoop()\r\n    {\r\n        ReadOnlyCollection<int> list = _list;\r\n        int sum = 0;\r\n        int count = list.Count;\r\n        for (int i = 0; i < count; i++)\r\n        {\r\n            sum += _list[i];\r\n        }\r\n        return sum;\r\n    }\r\n}<\/code><\/pre>\n
When asked “which of these will be faster”, the obvious answer is “SumForLoop<\/code>“. After all, SumEnumerable<\/code> is going to allocate an enumerator and has to make twice the number of interface calls (MoveNext<\/code>+Current<\/code> per iteration vs this[int]<\/code> per iteration). As it turns out, the obvious answer is also wrong. Here are the timings on my machine for .NET 9:<\/p>\n\n\n\n\n\n\n
Method<\/th>\nMean<\/th>\n<\/tr>\n<\/thead>\n
SumEnumerable<\/td>\n949.5 ns<\/td>\n<\/tr>\n
SumForLoop<\/td>\n1,932.7 ns<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
What the what?? If I change the ToArray<\/code> to instead be ToList<\/code>, however, the numbers are much more in line with our expectations.<\/p>\n\n\n\n\n\n\n
Method<\/th>\nMean<\/th>\n<\/tr>\n<\/thead>\n
SumEnumerable<\/td>\n1,542.0 ns<\/td>\n<\/tr>\n
SumForLoop<\/td>\n894.1 ns<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
So what’s going on here? It’s super subtle. First, it’s necessary to know that ReadOnlyCollection<T><\/code> just wraps an arbitrary IList<T><\/code>, the ReadOnlyCollection<T><\/code>‘s GetEnumerator()<\/code> returns _list.GetEnumerator()<\/code> (I’m ignoring for this discussion the special-case where the list is empty), and ReadOnlyCollection<T><\/code>‘s indexer just indexes into the IList<T><\/code>‘s indexer. So far presumably this all sounds like what you’d expect. But where things gets interesting is around what the JIT is able to devirtualize. In .NET 9, it struggles to devirtualize calls to the interface implementations specifically on T[]<\/code>, so it won’t devirtualize either the _list.GetEnumerator()<\/code> call nor the _list[index]<\/code> call. However, the enumerator that’s returned is just a normal type that implements IEnumerator<T><\/code>, and the JIT has no problem devirtualizing its MoveNext<\/code> and Current<\/code> members. Which means that we’re actually paying a lot more going through the indexer, because for N<\/code> elements, we’re having to make N<\/code> interface calls, whereas with the enumerator, we only need the one with GetEnumerator<\/code> interface call and then no more after that.<\/p>\n
Thankfully, this is now addressed in .NET 10. dotnet\/runtime#108153<\/a>, dotnet\/runtime#109209<\/a>, dotnet\/runtime#109237<\/a>, and dotnet\/runtime#116771<\/a> all make it possible for the JIT to devirtualize array’s interface method implementations. Now when we run the same benchmark (reverted back to using ToArray<\/code>), we get results much more in line with our expectations, with both benchmarks improving from .NET 9 to .NET 10, and with SumForLoop<\/code> on .NET 10 being the fastest.<\/p>\n\n\n\n\n\n\n\n\n\n
Method<\/th>\nRuntime<\/th>\nMean<\/th>\nRatio<\/th>\n<\/tr>\n<\/thead>\n
SumEnumerable<\/td>\n.NET 9.0<\/td>\n968.5 ns<\/td>\n1.00<\/td>\n<\/tr>\n
SumEnumerable<\/td>\n.NET 10.0<\/td>\n775.5 ns<\/td>\n0.80<\/td>\n<\/tr>\n
<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
SumForLoop<\/td>\n.NET 9.0<\/td>\n1,960.5 ns<\/td>\n1.00<\/td>\n<\/tr>\n
SumForLoop<\/td>\n.NET 10.0<\/td>\n624.6 ns<\/td>\n0.32<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
One of the really interesting things about this is how many libraries are implemented on the premise that it’s faster to use an IList<T><\/code>‘s indexer for iteration than it is to use its IEnumerable<T><\/code> for iteration, and that includes System.Linq<\/code>. All these years, where LINQ has had specialized code paths for working with IList<T><\/code> when possible, while in many cases it’s been a welcome optimization, in some<\/em> cases (such as when the concrete type is a ReadOnlyCollection<T><\/code>), it’s actually been a deoptimization.<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\nusing System.Collections.ObjectModel;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private ReadOnlyCollection<int> _list = new(Enumerable.Range(1, 1000).ToArray());\r\n\r\n    [Benchmark]\r\n    public int SkipTakeSum() => _list.Skip(100).Take(800).Sum();\r\n}<\/code><\/pre>\n\n\n\n\n\n\n
Method<\/th>\nRuntime<\/th>\nMean<\/th>\nRatio<\/th>\n<\/tr>\n<\/thead>\n
SkipTakeSum<\/td>\n.NET 9.0<\/td>\n3.525 us<\/td>\n1.00<\/td>\n<\/tr>\n
SkipTakeSum<\/td>\n.NET 10.0<\/td>\n1.773 us<\/td>\n0.50<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
Fixing devirtualization for array’s interface implementation then also has this transitive effect on LINQ.<\/p>\n
Guarded Devirtualization (GDV) is also improved in .NET 10, such as from dotnet\/runtime#116453<\/a> and dotnet\/runtime#109256<\/a>. With dynamic PGO<\/a>, the JIT is able to instrument a method’s compilation and then use the resulting profiling data as part of emitting an optimized version of the method. One of the things it can profile are which types are used in a virtual dispatch. If one type dominates, it can special-case that type in the code gen and emit a customized implementation specific to that type. That then enables devirtualization in that dedicated path, which is “guarded” by the relevant type check, hence “GDV”. In some cases, however, such as if a virtual call was being made in a shared generic context, GDV would not kick in. Now it will.<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\nusing System.Runtime.CompilerServices;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    [Benchmark]\r\n    public bool Test() => GenericEquals(\"abc\", \"abc\");\r\n\r\n    [MethodImpl(MethodImplOptions.NoInlining)]\r\n    private static bool GenericEquals<T>(T a, T b) => EqualityComparer<T>.Default.Equals(a, b);\r\n}<\/code><\/pre>\n\n\n\n\n\n\n
Method<\/th>\nRuntime<\/th>\nMean<\/th>\nRatio<\/th>\n<\/tr>\n<\/thead>\n
Test<\/td>\n.NET 9.0<\/td>\n2.816 ns<\/td>\n1.00<\/td>\n<\/tr>\n
Test<\/td>\n.NET 10.0<\/td>\n1.511 ns<\/td>\n0.54<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
dotnet\/runtime#110827<\/a> from @hez2010<\/a> also helps more methods to be inlined by doing another pass looking for opportunities after later phases of devirtualization. The JIT’s optimizations are split up into multiple phases; each phase can make improvements, and those improvements can expose additional opportunities. If those opportunities would only be capitalized on by a phase that already ran, they can be missed. But for phases that are relatively cheap to perform, such as doing a pass looking for additional inlining opportunities, those phases can be repeated once enough other optimization has happened that it’s likely productive to do so again.<\/p>\n
Bounds Checking<\/h3>\n
C# is a memory-safe language, an important aspect of modern programming languages. A key component of this is the inability to walk off the beginning or end of an array, string, or span. The runtime ensures that any such invalid attempt produces an exception, rather than being allowed to perform the invalid memory access. We can see what this looks like with a small benchmark:<\/p>\n
\/\/ dotnet run -c Release -f net10.0 --filter \"*\"\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private int[] _array = new int[3];\r\n\r\n    [Benchmark]\r\n    public int Read() => _array[2];\r\n}<\/code><\/pre>\n
This is a valid access: the _array<\/code> contains three elements, and the Read<\/code> method is reading its last element. However, the JIT can’t be 100% certain that this access is in bounds (something could have changed what’s in the _array<\/code> field to be a shorter array), and thus it needs to emit a check to ensure we’re not walking off the end of the array. Here’s what the generated assembly code for Read<\/code> looks like:<\/p>\n
; .NET 10\r\n; Tests.Read()\r\n       push      rax\r\n       mov       rax,[rdi+8]\r\n       cmp       dword ptr [rax+8],2\r\n       jbe       short M00_L00\r\n       mov       eax,[rax+18]\r\n       add       rsp,8\r\n       ret\r\nM00_L00:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 25<\/code><\/pre>\n
The this<\/code> reference is passed into the Read<\/code> instance method in the rdi<\/code> register, and the _array<\/code> field is at offset 8, so the mov rax,[rdi+8]<\/code> instruction is loading the address of the array into the rax<\/code> register. Then the cmp<\/code> is loading the value at offset 8 from that address; it so happens that’s where the length of the array is stored in the array object. So, this cmp<\/code> instruction is the bounds check; it’s comparing 2<\/code> against that length to ensure it’s in bounds. If the array were too short for this access, the next jbe<\/code> instruction would branch to the M00_L00<\/code> label, which calls the CORINFO_HELP_RNGCHKFAIL<\/code> helper function that throws an IndexOutOfRangeException<\/code>. Any time you see this pair of call CORINFO_HELP_RNGCHKFAIL<\/code>\/int 3<\/code> at the end of a method, there was at least one bounds check emitted by the JIT in that method.<\/p>\n
Of course, we not only want safety, we also want great performance, and it’d be terrible for performance if every single read from an array (or string or span) incurred such an additional check. As such, the JIT strives to avoid emitting these checks when they’d be redundant, when it can prove by construction that the accesses are safe. For example, let me tweak my benchmark slightly, moving the array from an instance field into a static readonly<\/code> field:<\/p>\n
\/\/ dotnet run -c Release -f net10.0 --filter \"*\"\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private static readonly int[] s_array = new int[3];\r\n\r\n    [Benchmark]\r\n    public int Read() => s_array[2];\r\n}<\/code><\/pre>\n
We now get this assembly:<\/p>\n
; .NET 10\r\n; Tests.Read()\r\n       mov       rax,705D5419FA20\r\n       mov       eax,[rax+18]\r\n       ret\r\n; Total bytes of code 14<\/code><\/pre>\n
The static readonly<\/code> field is immutable, arrays can’t be resized, and the JIT can guarantee that the field is initialized prior to generating the code for Read<\/code>. Therefore, when generating the code for Read<\/code>, it can know with certainty that the array is of length three, and we’re accessing the element at index two. Therefore, the specified array index is guaranteed to be within bounds, and there’s no need for a bounds check. We simply get two mov<\/code>s, the first mov<\/code> to load the address of the array (which, thanks to improvements in previous releases, is allocated on a heap that doesn’t need to be compacted such that the array lives at a fixed address), and the second mov<\/code> to read the int<\/code> value at the location of index two (these are int<\/code>s, so index two lives 2 * sizeof(int) = 8<\/code> bytes from the start of the array’s data, which itself on 64-bit is offset 16 bytes from the start of the array reference, for a total offset of 24 bytes, or in hex 0x18, hence the rax+18<\/code> in the disassembly).<\/p>\n
Every release of .NET, more and more opportunities are found and implemented to eschew bounds checks that were previously being generated. .NET 10 continues this trend.<\/p>\n
Our first example comes from dotnet\/runtime#109900<\/a>, which was inspired by the implementation of BitOperations.Log2<\/code>. The operation has intrinsic hardware support on many architectures, and generally BitOperations.Log2<\/code> will use one of the hardware intrinsics available to it for a very efficient implementation (e.g. Lscnt.LeadingZeroCount<\/code>, ArmBase.LeadingZeroCount<\/code>, or X86Base.BitScanReverse<\/code>), however as a fallback implementation it uses a lookup table. The lookup table has 32 elements, and the operation involves computing a uint<\/code> value and then shifting it down by 27 in order to get the top 5 bits. Any possible result is guaranteed to be a non-negative number less than 32, but indexing into the span with that result still produced a bounds check, and, as this is a critical path, “unsafe” code (meaning code that eschews the guardrails the runtime supplies by default) was then used to avoid the bounds check.<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\", \"value\")]\r\npublic partial class Tests\r\n{\r\n    [Benchmark]\r\n    [Arguments(42)]\r\n    public int Log2SoftwareFallback2(uint value)\r\n    {\r\n        ReadOnlySpan<byte> Log2DeBruijn =\r\n        [\r\n            00, 09, 01, 10, 13, 21, 02, 29,\r\n            11, 14, 16, 18, 22, 25, 03, 30,\r\n            08, 12, 20, 28, 15, 17, 24, 07,\r\n            19, 27, 23, 06, 26, 05, 04, 31\r\n        ];\r\n\r\n        value |= value >> 01;\r\n        value |= value >> 02;\r\n        value |= value >> 04;\r\n        value |= value >> 08;\r\n        value |= value >> 16;\r\n\r\n        return Log2DeBruijn[(int)((value * 0x07C4ACDDu) >> 27)];\r\n    }\r\n}<\/code><\/pre>\n
Now in .NET 10, the bounds check is gone (note the presence of the call CORINFO_HELP_RNGCHKFAIL<\/code> in the .NET 9 assembly and the lack of it in the .NET 10 assembly).<\/p>\n
; .NET 9\r\n; Tests.Log2SoftwareFallback2(UInt32)\r\n       push      rax\r\n       mov       eax,esi\r\n       shr       eax,1\r\n       or        esi,eax\r\n       mov       eax,esi\r\n       shr       eax,2\r\n       or        esi,eax\r\n       mov       eax,esi\r\n       shr       eax,4\r\n       or        esi,eax\r\n       mov       eax,esi\r\n       shr       eax,8\r\n       or        esi,eax\r\n       mov       eax,esi\r\n       shr       eax,10\r\n       or        eax,esi\r\n       imul      eax,7C4ACDD\r\n       shr       eax,1B\r\n       cmp       eax,20\r\n       jae       short M00_L00\r\n       mov       rcx,7913CA812E10\r\n       movzx     eax,byte ptr [rax+rcx]\r\n       add       rsp,8\r\n       ret\r\nM00_L00:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 74\r\n\r\n; .NET 10\r\n; Tests.Log2SoftwareFallback2(UInt32)\r\n       mov       eax,esi\r\n       shr       eax,1\r\n       or        esi,eax\r\n       mov       eax,esi\r\n       shr       eax,2\r\n       or        esi,eax\r\n       mov       eax,esi\r\n       shr       eax,4\r\n       or        esi,eax\r\n       mov       eax,esi\r\n       shr       eax,8\r\n       or        esi,eax\r\n       mov       eax,esi\r\n       shr       eax,10\r\n       or        eax,esi\r\n       imul      eax,7C4ACDD\r\n       shr       eax,1B\r\n       mov       rcx,7CA298325E10\r\n       movzx     eax,byte ptr [rcx+rax]\r\n       ret\r\n; Total bytes of code 58<\/code><\/pre>\n
This improvement then enabled dotnet\/runtime#118560<\/a> to simplify the code in the real Log2SoftwareFallback<\/code>, avoiding manual use of unsafe constructs.<\/p>\n
dotnet\/runtime#113790<\/a> implements a similar case, where the result of a mathematical operation is guaranteed to be in bounds. In this case, it’s the result of Log2<\/code>. The change teaches the JIT to understand the maximum possible value that Log2<\/code> could produce, and if that maximum is in bounds, then any result is guaranteed to be in bounds as well.<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\", \"value\")]\r\npublic partial class Tests\r\n{\r\n    [Benchmark]\r\n    [Arguments(12345)]\r\n    public nint CountDigits(ulong value)\r\n    {\r\n        ReadOnlySpan<byte> log2ToPow10 =\r\n        [\r\n            1,  1,  1,  2,  2,  2,  3,  3,  3,  4,  4,  4,  4,  5,  5,  5,\r\n            6,  6,  6,  7,  7,  7,  7,  8,  8,  8,  9,  9,  9,  10, 10, 10,\r\n            10, 11, 11, 11, 12, 12, 12, 13, 13, 13, 13, 14, 14, 14, 15, 15,\r\n            15, 16, 16, 16, 16, 17, 17, 17, 18, 18, 18, 19, 19, 19, 19, 20\r\n        ];\r\n\r\n        return log2ToPow10[(int)ulong.Log2(value)];\r\n    }\r\n}<\/code><\/pre>\n
We can see the bounds check present in the .NET 9 output and absent in the .NET 10 output:<\/p>\n
; .NET 9\r\n; Tests.CountDigits(UInt64)\r\n       push      rax\r\n       or        rsi,1\r\n       xor       eax,eax\r\n       lzcnt     rax,rsi\r\n       xor       eax,3F\r\n       cmp       eax,40\r\n       jae       short M00_L00\r\n       mov       rcx,7C2D0A213DF8\r\n       movzx     eax,byte ptr [rax+rcx]\r\n       add       rsp,8\r\n       ret\r\nM00_L00:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 45\r\n\r\n; .NET 10\r\n; Tests.CountDigits(UInt64)\r\n       or        rsi,1\r\n       xor       eax,eax\r\n       lzcnt     rax,rsi\r\n       xor       eax,3F\r\n       mov       rcx,71EFA9400DF8\r\n       movzx     eax,byte ptr [rcx+rax]\r\n       ret\r\n; Total bytes of code 29<\/code><\/pre>\n
My choice of benchmark in this case was not coincidental. This pattern shows up in the FormattingHelpers.CountDigits<\/code> internal method that’s used by the core primitive types in their ToString<\/code> and TryFormat<\/code> implementations, in order to determine how much space will be needed to store rendered digits for a number. As with the previous example, this routine is considered core enough that it was using unsafe code to avoid the bounds check. With this fix, the code was able to be changed back to using a simple span access, and even with the simpler code, it’s now also faster.<\/p>\n
Now, consider this code:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\", \"ids\")]\r\npublic partial class Tests\r\n{\r\n    public IEnumerable<int[]> Ids { get; } = [[1, 2, 3, 4, 5, 1]];\r\n\r\n    [Benchmark]\r\n    [ArgumentsSource(nameof(Ids))]\r\n    public bool StartAndEndAreSame(int[] ids) => ids[0] == ids[^1];\r\n}<\/code><\/pre>\n
I have a method that’s accepting an int[]<\/code> and checking to see whether it starts and ends with the same value. The JIT has no way of knowing whether the int[]<\/code> is empty or not, so it does<\/em> need a bounds check; otherwise, accessing ids[0]<\/code> could walk off the end of the array. However, this is what we see on .NET 9:<\/p>\n
; .NET 9\r\n; Tests.StartAndEndAreSame(Int32[])\r\n       push      rax\r\n       mov       eax,[rsi+8]\r\n       test      eax,eax\r\n       je        short M00_L00\r\n       mov       ecx,[rsi+10]\r\n       lea       edx,[rax-1]\r\n       cmp       edx,eax\r\n       jae       short M00_L00\r\n       mov       eax,edx\r\n       cmp       ecx,[rsi+rax*4+10]\r\n       sete      al\r\n       movzx     eax,al\r\n       add       rsp,8\r\n       ret\r\nM00_L00:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 41<\/code><\/pre>\n
Note there are two jumps to the M00_L00<\/code> label that handles failed bounds checks… that’s because there are two bounds checks here, one for the start access and one for the end access. But that shouldn’t be necessary. ids[^1]<\/code> is the same as ids[ids.Length - 1]<\/code>. If the code has successfully accessed ids[0]<\/code>, that means the array is at least one element in length, and if it’s at least one element in length, ids[ids.Length - 1]<\/code> will always be in bounds. Thus, the second bounds check shouldn’t be needed. Indeed, thanks to dotnet\/runtime#116105<\/a>, this is what we now get on .NET 10 (one branch to M00_L00<\/code> instead of two):<\/p>\n
; .NET 10\r\n; Tests.StartAndEndAreSame(Int32[])\r\n       push      rax\r\n       mov       eax,[rsi+8]\r\n       test      eax,eax\r\n       je        short M00_L00\r\n       mov       ecx,[rsi+10]\r\n       dec       eax\r\n       cmp       ecx,[rsi+rax*4+10]\r\n       sete      al\r\n       movzx     eax,al\r\n       add       rsp,8\r\n       ret\r\nM00_L00:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 34<\/code><\/pre>\n
What’s really interesting to me here is the knock-on effect of having removed the bounds check. It didn’t just eliminate the cmp\/jae<\/code> pair of instructions that’s typical of a bounds check. The .NET 9 version of the code had this:<\/p>\n
lea edx,[rax-1]\r\ncmp edx,eax\r\njae short M00_L00\r\nmov eax,edx<\/code><\/pre>\n
At this point in the assembly, the rax<\/code> register is storing the length of the array. It’s calculating ids.Length - 1<\/code> and storing the result into edx<\/code>, and then checking to see whether ids.Length-1<\/code> is in bounds of ids.Length<\/code> (the only way it wouldn’t be is if the array were empty such that ids.Length-1<\/code> wrapped around to uint.MaxValue<\/code>); if it’s not, it jumps to the fail handler, and if it is, it stores the already computed ids.Length - 1<\/code> into eax<\/code>. By removing the bounds check, we get rid of those two intervening instructions, leaving these:<\/p>\n
lea edx,[rax-1]\r\nmov eax,edx<\/code><\/pre>\n
which is a little silly, as this sequence is just computing a decrement, and as long as it’s ok that flags get modified, it could instead just be:<\/p>\n
dec eax<\/code><\/pre>\n
which, as you can see in the .NET 10 output, is exactly what .NET 10 now does.<\/p>\n
dotnet\/runtime#115980<\/a> addresses another case. Let’s say I have this method:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\", \"start\", \"text\")]\r\npublic partial class Tests\r\n{\r\n    [Benchmark]\r\n    [Arguments(\"abc\", \"abc.\")]\r\n    public bool IsFollowedByPeriod(string start, string text) =>\r\n        start.Length < text.Length && text[start.Length] == '.';\r\n}<\/code><\/pre>\n
We’re validating that one input’s length is less than the other, and then checking to see what comes immediately after it in the other. We know that string.Length<\/code> is immutable, so a bounds check here is redundant, but until .NET 10, the JIT couldn’t see that.<\/p>\n
; .NET 9\r\n; Tests.IsFollowedByPeriod(System.String, System.String)\r\n       push      rbp\r\n       mov       rbp,rsp\r\n       mov       eax,[rsi+8]\r\n       mov       ecx,[rdx+8]\r\n       cmp       eax,ecx\r\n       jge       short M00_L00\r\n       cmp       eax,ecx\r\n       jae       short M00_L01\r\n       cmp       word ptr [rdx+rax*2+0C],2E\r\n       sete      al\r\n       movzx     eax,al\r\n       pop       rbp\r\n       ret\r\nM00_L00:\r\n       xor       eax,eax\r\n       pop       rbp\r\n       ret\r\nM00_L01:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 42\r\n\r\n; .NET 10\r\n; Tests.IsFollowedByPeriod(System.String, System.String)\r\n       mov       eax,[rsi+8]\r\n       mov       ecx,[rdx+8]\r\n       cmp       eax,ecx\r\n       jge       short M00_L00\r\n       cmp       word ptr [rdx+rax*2+0C],2E\r\n       sete      al\r\n       movzx     eax,al\r\n       ret\r\nM00_L00:\r\n       xor       eax,eax\r\n       ret\r\n; Total bytes of code 26<\/code><\/pre>\n
The removal of the bounds check almost halves the size of the function. If we don’t need to do a bounds check, we get to elide the cmp\/jae<\/code>. Without that branch, nothing is targeting M00_L01<\/code>, and we can remove the call\/int<\/code> pair that were only necessary to support a bounds check. Then without the call<\/code> in M00_L01<\/code>, which was the only call<\/code> in the whole method, the prologue and epilogue can be elided, meaning we also don’t need the opening and closing push<\/code> and pop<\/code> instructions.<\/p>\n
dotnet\/runtime#113233<\/a> improved handling “assertions” (facts the JIT claims and based on which the JIT makes optimizations) to be less order dependent. In .NET 9, this code:<\/p>\n
static bool Test(ReadOnlySpan<char> span, int pos) =>\r\n    pos > 0 &&\r\n    pos <= span.Length - 42 &&\r\n    span[pos - 1] != '\\n';<\/code><\/pre>\n
was successfully removing the bounds check on the span access, but the following variant, which just switches the order of the first two conditions, was still incurring the bounds check.<\/p>\n
static bool Test(ReadOnlySpan<char> span, int pos) =>\r\n    pos <= span.Length - 42 &&\r\n    pos > 0 &&\r\n    span[pos - 1] != '\\n';<\/code><\/pre>\n
Note that both conditions contribute an assertion (fact) that need to be merged in order to know the bounds check can be avoided. Now in .NET 10, the bounds check is elided, regardless of the order.<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private string _s = new string('s', 100);\r\n    private int _pos = 10;\r\n\r\n    [Benchmark]\r\n    public bool Test()\r\n    {\r\n        string s = _s;\r\n        int pos = _pos;\r\n        return\r\n            pos <= s.Length - 42 &&\r\n            pos > 0 &&\r\n            s[pos - 1] != '\\n';\r\n    }\r\n}<\/code><\/pre>\n
; .NET 9\r\n; Tests.Test()\r\n       push      rbp\r\n       mov       rbp,rsp\r\n       mov       rax,[rdi+8]\r\n       mov       ecx,[rdi+10]\r\n       mov       edx,[rax+8]\r\n       lea       edi,[rdx-2A]\r\n       cmp       edi,ecx\r\n       jl        short M00_L00\r\n       test      ecx,ecx\r\n       jle       short M00_L00\r\n       dec       ecx\r\n       cmp       ecx,edx\r\n       jae       short M00_L01\r\n       cmp       word ptr [rax+rcx*2+0C],0A\r\n       setne     al\r\n       movzx     eax,al\r\n       pop       rbp\r\n       ret\r\nM00_L00:\r\n       xor       eax,eax\r\n       pop       rbp\r\n       ret\r\nM00_L01:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 55\r\n\r\n; .NET 10\r\n; Tests.Test()\r\n       push      rbp\r\n       mov       rbp,rsp\r\n       mov       rax,[rdi+8]\r\n       mov       ecx,[rdi+10]\r\n       mov       edx,[rax+8]\r\n       add       edx,0FFFFFFD6\r\n       cmp       edx,ecx\r\n       jl        short M00_L00\r\n       test      ecx,ecx\r\n       jle       short M00_L00\r\n       dec       ecx\r\n       cmp       word ptr [rax+rcx*2+0C],0A\r\n       setne     al\r\n       movzx     eax,al\r\n       pop       rbp\r\n       ret\r\nM00_L00:\r\n       xor       eax,eax\r\n       pop       rbp\r\n       ret\r\n; Total bytes of code 45<\/code><\/pre>\n
dotnet\/runtime#113862<\/a> addresses a similar case where assertions weren’t being handled as precisely as they could have been. Consider this code:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[MemoryDiagnoser(displayGenColumns: false)]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private int[] _arr = Enumerable.Range(0, 10).ToArray();\r\n\r\n    [Benchmark]\r\n    public int Sum()\r\n    {\r\n        int[] arr = _arr;\r\n        int sum = 0;\r\n\r\n        int i;\r\n        for (i = 0; i < arr.Length - 3; i += 4)\r\n        {\r\n            sum += arr[i + 0];\r\n            sum += arr[i + 1];\r\n            sum += arr[i + 2];\r\n            sum += arr[i + 3];\r\n        }\r\n\r\n        for (; i < arr.Length; i++)\r\n        {\r\n            sum += arr[i];\r\n        }\r\n\r\n        return sum;\r\n    }\r\n}<\/code><\/pre>\n
The Sum<\/code> method is trying to do manual loop unrolling. Rather than incurring a branch on each element, it’s handling four elements per iteration. Then, for the case where the length of the input isn’t evenly divisible by four, it’s handling the remaining elements in a separate loop. In .NET 9, the JIT successfully elides the bounds checks in the main unrolled loop:<\/p>\n
; .NET 9\r\n; Tests.Sum()\r\n       push      rbp\r\n       mov       rbp,rsp\r\n       mov       rax,[rdi+8]\r\n       xor       ecx,ecx\r\n       xor       edx,edx\r\n       mov       edi,[rax+8]\r\n       lea       esi,[rdi-3]\r\n       test      esi,esi\r\n       jle       short M00_L02\r\nM00_L00:\r\n       mov       r8d,edx\r\n       add       ecx,[rax+r8*4+10]\r\n       lea       r8d,[rdx+1]\r\n       add       ecx,[rax+r8*4+10]\r\n       lea       r8d,[rdx+2]\r\n       add       ecx,[rax+r8*4+10]\r\n       lea       r8d,[rdx+3]\r\n       add       ecx,[rax+r8*4+10]\r\n       add       edx,4\r\n       cmp       esi,edx\r\n       jg        short M00_L00\r\n       jmp       short M00_L02\r\nM00_L01:\r\n       cmp       edx,edi\r\n       jae       short M00_L03\r\n       mov       esi,edx\r\n       add       ecx,[rax+rsi*4+10]\r\n       inc       edx\r\nM00_L02:\r\n       cmp       edi,edx\r\n       jg        short M00_L01\r\n       mov       eax,ecx\r\n       pop       rbp\r\n       ret\r\nM00_L03:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 92<\/code><\/pre>\n
You can see this in the M00_L00<\/code> section, which has the five add<\/code> instructions (four for the summed elements, and one for the index). However, we still see the CORINFO_HELP_RNGCHKFAIL<\/code> at the end, indicating this method has a bounds check. That’s coming from the final loop, due to the JIT losing track of the fact that i<\/code> is guaranteed to be non-negative. Now in .NET 10, that bounds check is removed as well (again, just look for the lack of the CORINFO_HELP_RNGCHKFAIL<\/code> call).<\/p>\n
; .NET 10\r\n; Tests.Sum()\r\n       push      rbp\r\n       mov       rbp,rsp\r\n       mov       rax,[rdi+8]\r\n       xor       ecx,ecx\r\n       xor       edx,edx\r\n       mov       edi,[rax+8]\r\n       lea       esi,[rdi-3]\r\n       test      esi,esi\r\n       jle       short M00_L01\r\nM00_L00:\r\n       mov       r8d,edx\r\n       add       ecx,[rax+r8*4+10]\r\n       lea       r8d,[rdx+1]\r\n       add       ecx,[rax+r8*4+10]\r\n       lea       r8d,[rdx+2]\r\n       add       ecx,[rax+r8*4+10]\r\n       lea       r8d,[rdx+3]\r\n       add       ecx,[rax+r8*4+10]\r\n       add       edx,4\r\n       cmp       esi,edx\r\n       jg        short M00_L00\r\nM00_L01:\r\n       cmp       edi,edx\r\n       jle       short M00_L03\r\n       test      edx,edx\r\n       jl        short M00_L04\r\nM00_L02:\r\n       mov       esi,edx\r\n       add       ecx,[rax+rsi*4+10]\r\n       inc       edx\r\n       cmp       edi,edx\r\n       jg        short M00_L02\r\nM00_L03:\r\n       mov       eax,ecx\r\n       pop       rbp\r\n       ret\r\nM00_L04:\r\n       mov       esi,edx\r\n       add       ecx,[rax+rsi*4+10]\r\n       inc       edx\r\n       cmp       edi,edx\r\n       jg        short M00_L04\r\n       jmp       short M00_L03\r\n; Total bytes of code 102<\/code><\/pre>\n
Another nice improvement comes from dotnet\/runtime#112824<\/a>, which teaches the JIT to turn facts it already learned from earlier checks into concrete numeric ranges, and then use those ranges to fold away later relational tests and bounds checks. Consider this example:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\nusing System.Runtime.CompilerServices;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private int[] _array = new int[10];\r\n\r\n    [Benchmark]\r\n    public void Test() => SetAndSlice(_array);\r\n\r\n    [MethodImpl(MethodImplOptions.NoInlining)]\r\n    private static Span<int> SetAndSlice(Span<int> src)\r\n    {\r\n        src[5] = 42;\r\n        return src.Slice(4);\r\n    }\r\n}<\/code><\/pre>\n
We have to incur a bounds check for the src[5]<\/code>, as the JIT has no evidence that src<\/code> is at least six elements long. However, by the time we get to the Slice<\/code> call, we know the span has a length of at least six, or else writing into src[5]<\/code> would have failed. We can use that knowledge to remove the length check from within the Slice<\/code> call (note the removal of the call qword ptr [7F8DDB3A7810]<\/code>\/int 3<\/code> sequence, which is the manual length check and call to a throw helper method in Slice<\/code>).<\/p>\n
; .NET 9\r\n; Tests.SetAndSlice(System.Span`1<Int32>)\r\n       push      rbp\r\n       mov       rbp,rsp\r\n       cmp       esi,5\r\n       jbe       short M01_L01\r\n       mov       dword ptr [rdi+14],2A\r\n       cmp       esi,4\r\n       jb        short M01_L00\r\n       add       rdi,10\r\n       mov       rax,rdi\r\n       add       esi,0FFFFFFFC\r\n       mov       edx,esi\r\n       pop       rbp\r\n       ret\r\nM01_L00:\r\n       call      qword ptr [7F8DDB3A7810]\r\n       int       3\r\nM01_L01:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 48\r\n\r\n; .NET 10\r\n; Tests.SetAndSlice(System.Span`1<Int32>)\r\n       push      rax\r\n       cmp       esi,5\r\n       jbe       short M01_L00\r\n       mov       dword ptr [rdi+14],2A\r\n       lea       rax,[rdi+10]\r\n       lea       edx,[rsi-4]\r\n       add       rsp,8\r\n       ret\r\nM01_L00:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 31<\/code><\/pre>\n
Let’s look at one more, which has a very nice impact on bounds checking, even though technically the optimization is broader than just that. dotnet\/runtime#113998<\/a> creates assertions from switch<\/code> targets. This means that the body of a switch<\/code> case statement inherits facts about what was switched over based on what the case<\/code> was, e.g. in a case 3<\/code> for switch (x)<\/code>, the body of that case will now “know” that x<\/code> is three. This is great for very popular patterns with arrays, strings, and spans, where developers switch over the length and then index into available indices in the appropriate branches. Consider this:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\nusing System.Runtime.CompilerServices;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private int[] _array = [1, 2];\r\n\r\n    [Benchmark]\r\n    public int SumArray() => Sum(_array);\r\n\r\n    [MethodImpl(MethodImplOptions.NoInlining)]\r\n    public int Sum(ReadOnlySpan<int> span)\r\n    {\r\n        switch (span.Length)\r\n        {\r\n            case 0: return 0;\r\n            case 1: return span[0];\r\n            case 2: return span[0] + span[1];\r\n            case 3: return span[0] + span[1] + span[2];\r\n            default: return -1;\r\n        }\r\n    }\r\n}<\/code><\/pre>\n
On .NET 9, each of those six span<\/code> dereferences ends up with a bounds check:<\/p>\n
; .NET 9\r\n; Tests.Sum(System.ReadOnlySpan`1<Int32>)\r\n       push      rbp\r\n       mov       rbp,rsp\r\nM01_L00:\r\n       cmp       edx,2\r\n       jne       short M01_L02\r\n       test      edx,edx\r\n       je        short M01_L04\r\n       mov       eax,[rsi]\r\n       cmp       edx,1\r\n       jbe       short M01_L04\r\n       add       eax,[rsi+4]\r\nM01_L01:\r\n       pop       rbp\r\n       ret\r\nM01_L02:\r\n       cmp       edx,3\r\n       ja        short M01_L03\r\n       mov       eax,edx\r\n       lea       rcx,[783DA42091B8]\r\n       mov       ecx,[rcx+rax*4]\r\n       lea       rdi,[M01_L00]\r\n       add       rcx,rdi\r\n       jmp       rcx\r\nM01_L03:\r\n       mov       eax,0FFFFFFFF\r\n       pop       rbp\r\n       ret\r\n       test      edx,edx\r\n       je        short M01_L04\r\n       mov       eax,[rsi]\r\n       cmp       edx,1\r\n       jbe       short M01_L04\r\n       add       eax,[rsi+4]\r\n       cmp       edx,2\r\n       jbe       short M01_L04\r\n       add       eax,[rsi+8]\r\n       jmp       short M01_L01\r\n       test      edx,edx\r\n       je        short M01_L04\r\n       mov       eax,[rsi]\r\n       jmp       short M01_L01\r\n       xor       eax,eax\r\n       pop       rbp\r\n       ret\r\nM01_L04:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 103<\/code><\/pre>\n
You can see the tell-tale bounds check sign (CORINFO_HELP_RNGCHKFAIL<\/code>) under M01_L04<\/code>, and no fewer than six jumps targeting that label, one for each span[...]<\/code> access. But on .NET 10, we get this:<\/p>\n
; .NET 10\r\n; Tests.Sum(System.ReadOnlySpan`1<Int32>)\r\n       push      rbp\r\n       mov       rbp,rsp\r\nM01_L00:\r\n       cmp       edx,2\r\n       jne       short M01_L02\r\n       mov       eax,[rsi]\r\n       add       eax,[rsi+4]\r\nM01_L01:\r\n       pop       rbp\r\n       ret\r\nM01_L02:\r\n       cmp       edx,3\r\n       ja        short M01_L03\r\n       mov       eax,edx\r\n       lea       rcx,[72C15C0F8FD8]\r\n       mov       ecx,[rcx+rax*4]\r\n       lea       rdx,[M01_L00]\r\n       add       rcx,rdx\r\n       jmp       rcx\r\nM01_L03:\r\n       mov       eax,0FFFFFFFF\r\n       pop       rbp\r\n       ret\r\n       xor       eax,eax\r\n       pop       rbp\r\n       ret\r\n       mov       eax,[rsi]\r\n       jmp       short M01_L01\r\n       mov       eax,[rsi]\r\n       add       eax,[rsi+4]\r\n       add       eax,[rsi+8]\r\n       jmp       short M01_L01\r\n; Total bytes of code 70<\/code><\/pre>\n
The CORINFO_HELP_RNGCHKFAIL<\/code> and all the jumps to it have evaporated.<\/p>\n
Cloning<\/h3>\n
There are other ways the JIT can remove bounds checking even when it can’t prove statically that every individual access is safe. Consider this method:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private int[] _arr = new int[16];\r\n\r\n    [Benchmark]\r\n    public void Test()\r\n    {\r\n        int[] arr = _arr;\r\n        arr[0] = 2;\r\n        arr[1] = 3;\r\n        arr[2] = 5;\r\n        arr[3] = 8;\r\n        arr[4] = 13;\r\n        arr[5] = 21;\r\n        arr[6] = 34;\r\n        arr[7] = 55;\r\n    }\r\n}<\/code><\/pre>\n
Here’s the assembly code generated on .NET 9:<\/p>\n
; .NET 9\r\n; Tests.Test()\r\n       push      rax\r\n       mov       rax,[rdi+8]\r\n       mov       ecx,[rax+8]\r\n       test      ecx,ecx\r\n       je        short M00_L00\r\n       mov       dword ptr [rax+10],2\r\n       cmp       ecx,1\r\n       jbe       short M00_L00\r\n       mov       dword ptr [rax+14],3\r\n       cmp       ecx,2\r\n       jbe       short M00_L00\r\n       mov       dword ptr [rax+18],5\r\n       cmp       ecx,3\r\n       jbe       short M00_L00\r\n       mov       dword ptr [rax+1C],8\r\n       cmp       ecx,4\r\n       jbe       short M00_L00\r\n       mov       dword ptr [rax+20],0D\r\n       cmp       ecx,5\r\n       jbe       short M00_L00\r\n       mov       dword ptr [rax+24],15\r\n       cmp       ecx,6\r\n       jbe       short M00_L00\r\n       mov       dword ptr [rax+28],22\r\n       cmp       ecx,7\r\n       jbe       short M00_L00\r\n       mov       dword ptr [rax+2C],37\r\n       add       rsp,8\r\n       ret\r\nM00_L00:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 114<\/code><\/pre>\n
Even if you’re not proficient at reading assembly, the pattern should still be obvious. In the C# code, we have eight writes into the array, and in the assembly code, we have eight repetitions of the same pattern: cmp ecx,LENGTH<\/code> to compare the length of the array against the required LENGTH<\/code>, jbe short M00_L00<\/code> to jump to the CORINFO_HELP_RNGCHKFAIL<\/code> helper if the bounds check fails, and mov dword ptr [rax+OFFSET],VALUE<\/code> to store VALUE<\/code> into the array at byte offset OFFSET<\/code>. Inside the Test<\/code> method, the JIT can’t know how long _arr<\/code> is, so it must include bounds checks. Moreover, it must include all of the bounds checks, rather than coalescing them, because it is forbidden from introducing behavioral changes as part of optimizations. Imagine instead if it chose to coalesce all of the bounds checks into a single check, and emitted this method as if it were the equivalent of the following:<\/p>\n
if (arr.Length >= 8)\r\n{\r\n    arr[0] = 2;\r\n    arr[1] = 3;\r\n    arr[2] = 5;\r\n    arr[3] = 8;\r\n    arr[4] = 13;\r\n    arr[5] = 21;\r\n    arr[6] = 34;\r\n    arr[7] = 55;\r\n}\r\nelse\r\n{\r\n    throw new IndexOutOfRangeException();\r\n}<\/code><\/pre>\n
Now, let’s say the array was actually of length four. The original program would have filled the array with values [2, 3, 5, 8]<\/code> before throwing an exception, but this transformed code wouldn’t (there wouldn’t be any writes to the array). That’s an observable behavioral change. An enterprising developer could of course choose<\/em> to rewrite their code to avoid some of these checks, e.g.<\/p>\n
arr[7] = 55;\r\narr[0] = 2;\r\narr[1] = 3;\r\narr[2] = 5;\r\narr[3] = 8;\r\narr[4] = 13;\r\narr[5] = 21;\r\narr[6] = 34;<\/code><\/pre>\n
By moving the last store to the beginning, the developer has given the JIT extra knowledge. The JIT can now see that if<\/em> the first store succeeds, the rest are guaranteed to succeed as well, and the JIT will emit a single bounds check. But, again, that’s the developer choosing to change their program in a way the JIT must not. However, there are other things the JIT can<\/em> do. Imagine the JIT chose to rewrite the method like this instead:<\/p>\n
if (arr.Length >= 8)\r\n{\r\n    arr[0] = 2;\r\n    arr[1] = 3;\r\n    arr[2] = 5;\r\n    arr[3] = 8;\r\n    arr[4] = 13;\r\n    arr[5] = 21;\r\n    arr[6] = 34;\r\n    arr[7] = 55;\r\n}\r\nelse\r\n{\r\n    arr[0] = 2;\r\n    arr[1] = 3;\r\n    arr[2] = 5;\r\n    arr[3] = 8;\r\n    arr[4] = 13;\r\n    arr[5] = 21;\r\n    arr[6] = 34;\r\n    arr[7] = 55;\r\n}<\/code><\/pre>\n
To our C# sensibilities, that looks unnecessarily complicated; the if<\/code> and the else<\/code> block contain exactly<\/em> the same C# code. But, knowing what we now know about how the JIT can use known length information to elide bounds checks, it starts to make a bit more sense. Here’s what the JIT emits for this variant on .NET 9:<\/p>\n
; .NET 9\r\n; Tests.Test()\r\n       push      rbp\r\n       mov       rbp,rsp\r\n       mov       rax,[rdi+8]\r\n       mov       ecx,[rax+8]\r\n       cmp       ecx,8\r\n       jl        short M00_L00\r\n       mov       rcx,300000002\r\n       mov       [rax+10],rcx\r\n       mov       rcx,800000005\r\n       mov       [rax+18],rcx\r\n       mov       rcx,150000000D\r\n       mov       [rax+20],rcx\r\n       mov       rcx,3700000022\r\n       mov       [rax+28],rcx\r\n       pop       rbp\r\n       ret\r\nM00_L00:\r\n       test      ecx,ecx\r\n       je        short M00_L01\r\n       mov       dword ptr [rax+10],2\r\n       cmp       ecx,1\r\n       jbe       short M00_L01\r\n       mov       dword ptr [rax+14],3\r\n       cmp       ecx,2\r\n       jbe       short M00_L01\r\n       mov       dword ptr [rax+18],5\r\n       cmp       ecx,3\r\n       jbe       short M00_L01\r\n       mov       dword ptr [rax+1C],8\r\n       cmp       ecx,4\r\n       jbe       short M00_L01\r\n       mov       dword ptr [rax+20],0D\r\n       cmp       ecx,5\r\n       jbe       short M00_L01\r\n       mov       dword ptr [rax+24],15\r\n       cmp       ecx,6\r\n       jbe       short M00_L01\r\n       mov       dword ptr [rax+28],22\r\n       cmp       ecx,7\r\n       jbe       short M00_L01\r\n       mov       dword ptr [rax+2C],37\r\n       pop       rbp\r\n       ret\r\nM00_L01:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 177<\/code><\/pre>\n
The else<\/code> block is compiled to the M00_L00<\/code> label, which contains those same eight repeated blocks we saw earlier. But the if<\/code> block (above the M00_L00<\/code> label) is interesting. The only branch there is the initial array.Length >= 8<\/code> check I wrote in the C# code, emitted as the cmp ecx,8<\/code>\/jl short M00_L00<\/code> pair of instructions. The rest of the block is just mov<\/code> instructions (and you can see there are only four writes into the array rather than eight… the JIT has optimized the eight four-byte writes into four eight-byte writes). In our rewrite, we’ve manually cloned the code, so that in what we expect to be the vast, vast, vast majority case (presumably we wouldn’t have written the array writes in the first place if we thought they’d fail), we only incur the single length check, and then we have our “hopefully this is never needed” fallback case for the rare situation where it is. Of course, you shouldn’t (and shouldn’t need to) do such manual cloning. But, the JIT can do such cloning for you, and does.<\/p>\n
“Cloning” is an optimization long employed by the JIT, where it will do this kind of code duplication, typically of loops, when it believes that in doing so, it can heavily optimize a common case. Now in .NET 10, thanks to dotnet\/runtime#112595<\/a>, it can employ this same technique for these kinds of sequences of writes. Going back to our original benchmark, here’s what we now get on .NET 10:<\/p>\n
; .NET 10\r\n; Tests.Test()\r\n       push      rbp\r\n       mov       rbp,rsp\r\n       mov       rax,[rdi+8]\r\n       mov       ecx,[rax+8]\r\n       mov       edx,ecx\r\n       cmp       edx,7\r\n       jle       short M00_L01\r\n       mov       rdx,300000002\r\n       mov       [rax+10],rdx\r\n       mov       rcx,800000005\r\n       mov       [rax+18],rcx\r\n       mov       rcx,150000000D\r\n       mov       [rax+20],rcx\r\n       mov       rcx,3700000022\r\n       mov       [rax+28],rcx\r\nM00_L00:\r\n       pop       rbp\r\n       ret\r\nM00_L01:\r\n       test      edx,edx\r\n       je        short M00_L02\r\n       mov       dword ptr [rax+10],2\r\n       cmp       ecx,1\r\n       jbe       short M00_L02\r\n       mov       dword ptr [rax+14],3\r\n       cmp       ecx,2\r\n       jbe       short M00_L02\r\n       mov       dword ptr [rax+18],5\r\n       cmp       ecx,3\r\n       jbe       short M00_L02\r\n       mov       dword ptr [rax+1C],8\r\n       cmp       ecx,4\r\n       jbe       short M00_L02\r\n       mov       dword ptr [rax+20],0D\r\n       cmp       ecx,5\r\n       jbe       short M00_L02\r\n       mov       dword ptr [rax+24],15\r\n       cmp       ecx,6\r\n       jbe       short M00_L02\r\n       mov       dword ptr [rax+28],22\r\n       cmp       ecx,7\r\n       jbe       short M00_L02\r\n       mov       dword ptr [rax+2C],37\r\n       jmp       short M00_L00\r\nM00_L02:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 179<\/code><\/pre>\n
This structure looks almost identical to what we got when we manually cloned: the JIT has emitted the same code twice, except in one case, there are no bounds checks, and in the other case, there are all the bounds checks, and a single length check determines which path to follow. Pretty neat.<\/p>\n
As noted, the JIT has been doing cloning for years, in particular for loops over arrays. However, more and more code is being written against spans instead of arrays, and unfortunately this valuable optimization didn’t apply to spans. Now with dotnet\/runtime#113575<\/a>, it does! We can see this with a basic looping example:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private int[] _arr = new int[16];\r\n    private int _count = 8;\r\n\r\n    [Benchmark]\r\n    public void WithSpan()\r\n    {\r\n        Span<int> span = _arr;\r\n        int count = _count;\r\n\r\n        for (int i = 0; i < count; i++)\r\n        {\r\n            span[i] = i;\r\n        }\r\n    }\r\n\r\n    [Benchmark]\r\n    public void WithArray()\r\n    {\r\n        int[] arr = _arr;\r\n        int count = _count;\r\n\r\n        for (int i = 0; i < count; i++)\r\n        {\r\n            arr[i] = i;\r\n        }\r\n    }\r\n}<\/code><\/pre>\n
In both WithArray<\/code> and WithSpan<\/code>, we have the same loop, iterating from 0 to a _count<\/code> with an unknown relationship to the length of _arr<\/code>, so there has to be some kind of bounds checking emitted. Here’s what we get on .NET 9 for WithSpan<\/code>:<\/p>\n
; .NET 9\r\n; Tests.WithSpan()\r\n       push      rbp\r\n       mov       rbp,rsp\r\n       mov       rax,[rdi+8]\r\n       test      rax,rax\r\n       je        short M00_L03\r\n       lea       rcx,[rax+10]\r\n       mov       eax,[rax+8]\r\nM00_L00:\r\n       mov       edi,[rdi+10]\r\n       xor       edx,edx\r\n       test      edi,edi\r\n       jle       short M00_L02\r\n       nop       dword ptr [rax]\r\nM00_L01:\r\n       cmp       edx,eax\r\n       jae       short M00_L04\r\n       mov       [rcx+rdx*4],edx\r\n       inc       edx\r\n       cmp       edx,edi\r\n       jl        short M00_L01\r\nM00_L02:\r\n       pop       rbp\r\n       ret\r\nM00_L03:\r\n       xor       ecx,ecx\r\n       xor       eax,eax\r\n       jmp       short M00_L00\r\nM00_L04:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 59<\/code><\/pre>\n
There’s some upfront assembly here associated with loading _array<\/code> into a span, loading _count<\/code>, and checking to see whether the count is 0 (in which case the whole loop can be skipped). Then the core of the loop is at M00_L01<\/code>, which is repeatedly checking edx<\/code> (which contains i<\/code>) against the length of the span (in eax<\/code>), jumping to CORINFO_HELP_RNGCHKFAIL<\/code> if it’s an out-of-bounds access, writing edx<\/code> (i<\/code>) into the span at the next position, bumping up i<\/code>, and then jumping back to M00_L01<\/code> to keep iterating if i<\/code> is still less than count<\/code> (stored in edi<\/code>). In other words, we have two checks per iteration: is i<\/code> still within the bounds of the span, and is i<\/code> still less than count<\/code>. Now here’s what we get on .NET 9 for WithArray<\/code>:<\/p>\n
; .NET 9\r\n; Tests.WithArray()\r\n       push      rbp\r\n       mov       rbp,rsp\r\n       mov       rax,[rdi+8]\r\n       mov       ecx,[rdi+10]\r\n       xor       edx,edx\r\n       test      ecx,ecx\r\n       jle       short M00_L01\r\n       test      rax,rax\r\n       je        short M00_L02\r\n       cmp       [rax+8],ecx\r\n       jl        short M00_L02\r\n       nop       dword ptr [rax+rax]\r\nM00_L00:\r\n       mov       edi,edx\r\n       mov       [rax+rdi*4+10],edx\r\n       inc       edx\r\n       cmp       edx,ecx\r\n       jl        short M00_L00\r\nM00_L01:\r\n       pop       rbp\r\n       ret\r\nM00_L02:\r\n       cmp       edx,[rax+8]\r\n       jae       short M00_L03\r\n       mov       edi,edx\r\n       mov       [rax+rdi*4+10],edx\r\n       inc       edx\r\n       cmp       edx,ecx\r\n       jl        short M00_L02\r\n       jmp       short M00_L01\r\nM00_L03:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 71<\/code><\/pre>\n
Here, label M00_L02<\/code> looks very similar to the loop we just saw in WithSpan<\/code>, incurring both the check against count<\/code> and the bounds check on every iteration. But note section M00_L00<\/code>: it’s a clone of the same loop, still with the cmp edx,ecx<\/code> that checks i<\/code> against count<\/code> on each iteration, but no additional bounds checking in sight. The JIT has cloned the loop, specializing one to not have bounds checks, and then in the upfront section, it determines which path to follow based on a single check against the array’s length (cmp [rax+8],ecx<\/code>\/jl short M00_L02<\/code>). Now in .NET 10, here’s what we get for WithSpan<\/code>:<\/p>\n
; .NET 10\r\n; Tests.WithSpan()\r\n       push      rbp\r\n       mov       rbp,rsp\r\n       mov       rax,[rdi+8]\r\n       test      rax,rax\r\n       je        short M00_L04\r\n       lea       rcx,[rax+10]\r\n       mov       eax,[rax+8]\r\nM00_L00:\r\n       mov       edx,[rdi+10]\r\n       xor       edi,edi\r\n       test      edx,edx\r\n       jle       short M00_L02\r\n       cmp       edx,eax\r\n       jg        short M00_L03\r\nM00_L01:\r\n       mov       eax,edi\r\n       mov       [rcx+rax*4],edi\r\n       inc       edi\r\n       cmp       edi,edx\r\n       jl        short M00_L01\r\nM00_L02:\r\n       pop       rbp\r\n       ret\r\nM00_L03:\r\n       cmp       edi,eax\r\n       jae       short M00_L05\r\n       mov       esi,edi\r\n       mov       [rcx+rsi*4],edi\r\n       inc       edi\r\n       cmp       edi,edx\r\n       jl        short M00_L03\r\n       jmp       short M00_L02\r\nM00_L04:\r\n       xor       ecx,ecx\r\n       xor       eax,eax\r\n       jmp       short M00_L00\r\nM00_L05:\r\n       call      CORINFO_HELP_RNGCHKFAIL\r\n       int       3\r\n; Total bytes of code 75<\/code><\/pre>\n
As with WithArray<\/code> in .NET 9, WithSpan<\/code> for .NET 10 has the loop cloned, with the M00_L03<\/code> block containing the bounds check on each iteration, and the M00_L01<\/code> block eliding the bounds check on each iteration.<\/p>\n
The JIT gains more cloning abilities in .NET 10, as well. dotnet\/runtime#110020<\/a>, dotnet\/runtime#108604<\/a>, and dotnet\/runtime#110483<\/a> make it possible for the JIT to clone try\/finally<\/code> blocks, whereas previously it would immediately bail out of cloning any regions containing such constructs. This might seem niche, but it’s actually quite valuable when you consider that foreach<\/code>‘ing over an enumerable typically involves a hidden try<\/code>\/finally<\/code> for the finally<\/code> to call the enumerator’s Dispose<\/code>.<\/p>\n
Many of these different optimizations interact with each other. Dynamic PGO triggers a form of cloning, as part of the guarded devirtualization (GDV) mentioned earlier: if the instrumentation data reveals that a particular virtual call is generally performed on an instance of a specific type, the JIT can clone the resulting code into one path specific to that type and another path that handles any type. That then enables the specific-type code path to devirtualize the call and possibly inline it. And if it inlines it, that then provides more opportunities for the JIT to see that an object doesn’t escape, and potentially stack allocate it.\ndotnet\/runtime#111473<\/a>,\ndotnet\/runtime#116978<\/a>, dotnet\/runtime#116992<\/a>,\ndotnet\/runtime#117222<\/a>, and dotnet\/runtime#117295<\/a> enable that, enhancing escape analysis to determine if an object only escapes when such a generated type test fails (when the target object isn’t of the expected common type).<\/p>\n
I want to pause for a moment, because my words thus far aren’t nearly enthusiastic enough to highlight the magnitude of what this enables. The dotnet\/runtime<\/code> repo uses an automated performance analysis system which flags when benchmarks significantly improve or regress and ties those changes back to the responsible PR. This is what it looked like for this PR:\n\"Conditional\nWe can see why this is so good from a simple example:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\nusing System.Runtime.CompilerServices;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[MemoryDiagnoser(displayGenColumns: false)]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private int[] _values = Enumerable.Range(1, 100).ToArray();\r\n\r\n    [Benchmark]\r\n    public int Sum() => Sum(_values);\r\n\r\n    [MethodImpl(MethodImplOptions.NoInlining)]\r\n    private static int Sum(IEnumerable<int> values)\r\n    {\r\n        int sum = 0;\r\n        foreach (int value in values)\r\n        {\r\n            sum += value;\r\n        }\r\n        return sum;\r\n    }\r\n}<\/code><\/pre>\n
With dynamic PGO, the instrumented code for Sum<\/code> will see that values<\/code> is generally an int[]<\/code>, and it’ll be able to emit a specialized code path in the optimized Sum<\/code> implementation for when it is. And then with this ability to do conditional escape analysis, for the common path the JIT can see that the resulting GetEnumerator<\/code> produces an IEnumerator<int><\/code> that never escapes, such that along with all of the relevant methods being devirtualized and inlined, the enumerator can be stack allocated.<\/p>\n\n\n\n\n\n\n
Method<\/th>\nRuntime<\/th>\nMean<\/th>\nRatio<\/th>\nAllocated<\/th>\nAlloc Ratio<\/th>\n<\/tr>\n<\/thead>\n
Sum<\/td>\n.NET 9.0<\/td>\n109.86 ns<\/td>\n1.00<\/td>\n32 B<\/td>\n1.00<\/td>\n<\/tr>\n
Sum<\/td>\n.NET 10.0<\/td>\n35.45 ns<\/td>\n0.32<\/td>\n–<\/td>\n0.00<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
Just think about how many places in your apps and services you enumerate collections like this, and you can see why it’s such an exciting improvement. Note that these cases don’t always even require PGO. Consider a case like this:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[MemoryDiagnoser(displayGenColumns: false)]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private static readonly IEnumerable<int> s_values = new int[] { 1, 2, 3, 4, 5 };\r\n\r\n    [Benchmark]\r\n    public int Sum()\r\n    {\r\n        int sum = 0;\r\n        foreach (int value in s_values)\r\n        {\r\n            sum += value;\r\n        }\r\n        return sum;\r\n    }\r\n}<\/code><\/pre>\n
Here, the JIT can see that even though the s_values<\/code> is typed as IEnumerable<int><\/code>, it’s always actually an int[]<\/code>. In that case, dotnet\/runtime#111948<\/a> enables the return type to be retyped in the JIT as int[]<\/code> and the enumerator can be stack allocated.<\/p>\n\n\n\n\n\n\n
Method<\/th>\nRuntime<\/th>\nMean<\/th>\nRatio<\/th>\nAllocated<\/th>\nAlloc Ratio<\/th>\n<\/tr>\n<\/thead>\n
Sum<\/td>\n.NET 9.0<\/td>\n16.341 ns<\/td>\n1.00<\/td>\n32 B<\/td>\n1.00<\/td>\n<\/tr>\n
Sum<\/td>\n.NET 10.0<\/td>\n2.059 ns<\/td>\n0.13<\/td>\n–<\/td>\n0.00<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
Of course, too much cloning can be a bad thing, in particular as it increases code size. dotnet\/runtime#108771<\/a> employs a heuristic to determine whether loops that can<\/em> be cloned should<\/em> be cloned; the larger the loop, the less likely it’ll be to be cloned.<\/p>\n
Inlining<\/h3>\n
“Inlining”, which replaces a call to a function with a copy of that function’s implementation, has always been a critically important optimization. It’s easy to think about the benefits of inlining as just being about avoiding the overhead of a call, and while that can be meaningful (especially when considering security mechanisms like Intel’s Control-Flow Enforcement Technology, which slightly increases the cost of calls), generally the most benefit from inlining comes from knock-on benefits. Just as a simple example, if you have code like:<\/p>\n
int i = Divide(10, 5);\r\n\r\nstatic int Divide(int n, int d) => n \/ d;<\/code><\/pre>\n
if Divide<\/code> doesn’t get inlined, then when Divide<\/code> is called, it’ll need to perform the actual idiv<\/code>, which is a relatively expensive operation. In contrast, if Divide<\/code> is inlined, then the call site becomes:<\/p>\n
int i = 10 \/ 5;<\/code><\/pre>\n
which can be evaluated at compile time and becomes just:<\/p>\n
int i = 2;<\/code><\/pre>\n
More compelling examples were already seen throughout the discussion of escape analysis and stack allocation, which depend heavily on the ability to inline methods. Given the increased importance of inlining, it’s gotten even more focus in .NET 10.<\/p>\n
Some of the .NET work related to inlining is about enabling more kinds of things to be inlined. Historically, a variety of constructs present in a method would prevent that method from even being considered for inlining. Arguably the most well known of these is exception handling: methods with exception handling clauses, e.g. try\/catch<\/code> or try\/finally<\/code>, would not be inlined. Even a simple method like M<\/code> in this example:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[MemoryDiagnoser(displayGenColumns: false)]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private readonly object _o = new();\r\n\r\n    [Benchmark]\r\n    public int Test()\r\n    {\r\n        M(_o);\r\n        return 42;\r\n    }\r\n\r\n    private static void M(object o)\r\n    {\r\n        Monitor.Enter(o);\r\n        try\r\n        {\r\n        }\r\n        finally\r\n        {\r\n            Monitor.Exit(o);\r\n        }\r\n    }\r\n}<\/code><\/pre>\n
does not get inlined on .NET 9:<\/p>\n
; .NET 9\r\n; Tests.Test()\r\n       push      rax\r\n       mov       rdi,[rdi+8]\r\n       call      qword ptr [78F199864EE8]; Tests.M(System.Object)\r\n       mov       eax,2A\r\n       add       rsp,8\r\n       ret\r\n; Total bytes of code 21<\/code><\/pre>\n
But with a plethora of PRs, in particular dotnet\/runtime#112968<\/a>, dotnet\/runtime#113023<\/a>, dotnet\/runtime#113497<\/a>, and dotnet\/runtime#112998<\/a>, methods containing try\/finally<\/code> are no longer blocked from inlining (try\/catch<\/code> regions are still a challenge). For the same benchmark on .NET 10, we now get this assembly:<\/p>\n
; .NET 10\r\n; Tests.Test()\r\n       push      rbp\r\n       push      rbx\r\n       push      rax\r\n       lea       rbp,[rsp+10]\r\n       mov       rbx,[rdi+8]\r\n       test      rbx,rbx\r\n       je        short M00_L03\r\n       mov       rdi,rbx\r\n       call      00007920A0EE65E0\r\n       test      eax,eax\r\n       je        short M00_L02\r\nM00_L00:\r\n       mov       rdi,rbx\r\n       call      00007920A0EE6D50\r\n       test      eax,eax\r\n       jne       short M00_L04\r\nM00_L01:\r\n       mov       eax,2A\r\n       add       rsp,8\r\n       pop       rbx\r\n       pop       rbp\r\n       ret\r\nM00_L02:\r\n       mov       rdi,rbx\r\n       call      qword ptr [79202393C1F8]\r\n       jmp       short M00_L00\r\nM00_L03:\r\n       xor       edi,edi\r\n       call      qword ptr [79202393C1C8]\r\n       int       3\r\nM00_L04:\r\n       mov       edi,eax\r\n       mov       rsi,rbx\r\n       call      qword ptr [79202393C1E0]\r\n       jmp       short M00_L01\r\n; Total bytes of code 86<\/code><\/pre>\n
The details of the assembly don’t matter, other than it’s a whole lot more than was there before, because we’re now looking in large part at the implementation of M<\/code>. In addition to methods with try\/finally<\/code> now being inlineable, other improvements have also been made around exception handling. For example, dotnet\/runtime#110273<\/a> and dotnet\/runtime#110464<\/a> enable the removal of try\/catch<\/code> and try\/fault<\/code> blocks if it can prove the try<\/code> block can’t possibly throw. Consider this:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\", \"i\")]\r\npublic partial class Tests\r\n{\r\n    [Benchmark]\r\n    [Arguments(42)]\r\n    public int Test(int i)\r\n    {\r\n        try\r\n        {\r\n            i++;\r\n        }\r\n        catch\r\n        {\r\n            Console.WriteLine(\"Exception caught\");\r\n        }\r\n\r\n        return i;\r\n    }\r\n}<\/code><\/pre>\n
There’s nothing the try<\/code> block here can do that will result in an exception being thrown (assuming the developer hasn’t enabled checked arithmetic, in which case it could possibly throw an OverflowException<\/code>), yet on .NET 9 we get this assembly:<\/p>\n
; .NET 9\r\n; Tests.Test(Int32)\r\n       push      rbp\r\n       sub       rsp,10\r\n       lea       rbp,[rsp+10]\r\n       mov       [rbp-10],rsp\r\n       mov       [rbp-4],esi\r\n       mov       eax,[rbp-4]\r\n       inc       eax\r\n       mov       [rbp-4],eax\r\nM00_L00:\r\n       mov       eax,[rbp-4]\r\n       add       rsp,10\r\n       pop       rbp\r\n       ret\r\n       push      rbp\r\n       sub       rsp,10\r\n       mov       rbp,[rdi]\r\n       mov       [rsp],rbp\r\n       lea       rbp,[rbp+10]\r\n       mov       rdi,784B08950018\r\n       call      qword ptr [784B0DE44EE8]\r\n       lea       rax,[M00_L00]\r\n       add       rsp,10\r\n       pop       rbp\r\n       ret\r\n; Total bytes of code 79<\/code><\/pre>\n
Now on .NET 10, the JIT is able to elide the catch<\/code> and remove all ceremony related to the try<\/code> because it can see that ceremony is pointless overhead.<\/p>\n
; .NET 10\r\n; Tests.Test(Int32)\r\n       lea       eax,[rsi+1]\r\n       ret\r\n; Total bytes of code 4<\/code><\/pre>\n
That’s true even when the contents of the try<\/code> calls into other methods that are then inlined, exposing their contents to the JIT’s analysis.<\/p>\n
(As an aside, the JIT was already able to remove try\/finally<\/code> when the finally<\/code> was empty, but dotnet\/runtime#108003<\/a> catches even more cases of checking for empty finally<\/code>s again after most other optimizations have been run, in case they revealed additional empty blocks.)<\/p>\n
Another example is “GVM”. Previously, any method that called a GVM, or generic virtual method (a virtual method with a generic type parameter), would be blocked from being inlined.<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[MemoryDiagnoser(displayGenColumns: false)]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private Base _base = new();\r\n\r\n    [Benchmark]\r\n    public int Test()\r\n    {\r\n        M();\r\n        return 42;\r\n    }\r\n\r\n    private void M() => _base.M<object>();\r\n}\r\n\r\nclass Base\r\n{\r\n    public virtual void M<T>() { }\r\n}<\/code><\/pre>\n
On .NET 9, the above results in this assembly:<\/p>\n
; .NET 9\r\n; Tests.Test()\r\n       push      rax\r\n       call      qword ptr [728ED5664FD8]; Tests.M()\r\n       mov       eax,2A\r\n       add       rsp,8\r\n       ret\r\n; Total bytes of code 17<\/code><\/pre>\n
Now on .NET 10, with dotnet\/runtime#116773<\/a>, M<\/code> can now be inlined.<\/p>\n
; .NET 10\r\n; Tests.Test()\r\n       push      rbp\r\n       push      rbx\r\n       push      rax\r\n       lea       rbp,[rsp+10]\r\n       mov       rbx,[rdi+8]\r\n       mov       rdi,rbx\r\n       mov       rsi,offset MT_Base\r\n       mov       rdx,78034C95D2A0\r\n       call      System.Runtime.CompilerServices.VirtualDispatchHelpers.VirtualFunctionPointer(System.Object, IntPtr, IntPtr)\r\n       mov       rdi,rbx\r\n       call      rax\r\n       mov       eax,2A\r\n       add       rsp,8\r\n       pop       rbx\r\n       pop       rbp\r\n       ret\r\n; Total bytes of code 57<\/code><\/pre>\n
Another area of investment with inlining is to do with the heuristics around when methods should be inlined. Just inlining everything would be bad; inlining copies code, which results in more code, which can have significant negative repercussions. For example, inlining’s increased code size puts more pressure on caches. Processors have an instruction cache, a small amount of super fast memory in a CPU that stores recently used instructions, making them really fast to access again the next time they’re needed (such as the next iteration through a loop, or the next time that same function is called). Consider a method M<\/code>, and 100 call sites to M<\/code> that are all being accessed. If all of those share the same instructions for M<\/code>, because the 100 call sites are all actually calling M<\/code>, the instruction cache will only need to load M<\/code>‘s instructions once. If all of those 100 call sites each have their own copy of M<\/code>‘s instructions, then all 100 copies will separately be loaded through the cache, fighting with each other and other instructions for residence. The less likely it is that instructions are in the cache, the more likely it is that the CPU will stall waiting for the instructions to be loaded from memory.<\/p>\n
For this reason, the JIT needs to be careful what it inlines. It tries hard to avoid inlining anything that won’t benefit (e.g. a larger method whose instructions won’t be materially influenced by the caller’s context) while also trying hard to inline anything that will materially benefit (e.g. small functions where the code required to call the function is similar in size to the contents of the function, functions with instructions that could be materially impacted by information from the call site, etc.) As part of these heuristics, the JIT has the notion of “boosts,” where observations it makes about things methods do boost the chances of that method being inlined. dotnet\/runtime#114806<\/a> gives a boost to methods that appear to be returning new arrays of a small, fixed length; if those arrays can instead be allocated in the caller’s frame, the JIT might then be able to discover they don’t escape and enable them to be stack allocated. dotnet\/runtime#110596<\/a> similarly looks for boxing, as the caller could possibly instead avoid the box entirely.<\/p>\n
For the same purpose (and also just to minimize time spent performing compilation), the JIT also maintains a budget for how much it allows to be inlined into a method compilation… once it hits that budget, it might stop inlining anything. The budgeting scheme overall works ok<\/em>, however in certain circumstances it can run out of budget at very inopportune times, for example doing a lot of inlining at top-level call sites but then running out of budget by the time it gets to small methods that are critically-important to inline for good performance. To help mitigate these scenarios, dotnet\/runtime#114191<\/a> and dotnet\/runtime#118641<\/a> more than double the JIT’s default inlining budget.<\/p>\n
The JIT also pays a lot of attention to the number of local variables (e.g. parameters\/locals explicitly in the IL, JIT-created temporary locals, promoted struct fields, etc.) it tracks. To avoid creating too many, the JIT would stop inlining once it was already tracking 512. But as other changes have made inlining more aggressive, this (strangely hardcoded) limit gets hit more often, leaving very valuable inlinees out in the cold. dotnet\/runtime#118515<\/a> removed this fixed limit and instead ties it to a large percentage of the number of locals the JIT is allowed to track (by default, this ends up almost doubling the limit used by the inliner).<\/p>\n
Constant Folding<\/h3>\n
Constant folding is a compiler’s ability to perform operations, typically math, at compile-time rather than at run-time: given multiple constants and an expressed relationship between them, the compiler can “fold” those constants together into a new constant. So, if you have the C# code int M(int i) => i + 2 * 3;<\/code>, the C# compiler does constant folding and emits that into your compilation as if you’d written int M(int i) => i + 6;<\/code>. The JIT can and does also do constant folding, which is valuable especially when it’s based on information not available to the C# compiler. For example, the JIT can treat static readonly<\/code> fields or IntPtr.Size<\/code> or Vector128<T>.Count<\/code> as constants. And the JIT can do folding across inlines. For example, if you have:<\/p>\n
int M1(int i) => i + M2(2 * 3);\r\nint M2(int j) => j * Environment.ProcessorCount;<\/code><\/pre>\n
the C# compiler will only be able to fold the 2 * 3<\/code>, and will emit the equivalent of:<\/p>\n
int M1(int i) => i + M2(6);\r\nint M2(int j) => j * Environment.ProcessorCount;<\/code><\/pre>\n
but when compiling M1<\/code>, the JIT can inline M2<\/code> and treat ProcessorCount<\/code> as a constant (on my machine it’s 16), and produce the following assembly code for M1<\/code>:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\"\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\", \"i\")]\r\npublic partial class Tests\r\n{\r\n    [Benchmark]\r\n    [Arguments(42)]\r\n    public int M1(int i) => i + M2(6);\r\n\r\n    private int M2(int j) => j * Environment.ProcessorCount;\r\n}<\/code><\/pre>\n
; .NET 9\r\n; Tests.M1(Int32)\r\n       lea       eax,[rsi+60]\r\n       ret\r\n; Total bytes of code 4<\/code><\/pre>\n
That’s as if the code for M1<\/code> had been public int M1(int i) => i + 96;<\/code> (the displayed assembly renders hexadecimal, so the 60<\/code> is hexadecimal 0x60<\/code> and thus decimal 96<\/code>).<\/p>\n
Or consider:<\/p>\n
string M() => GetString() ?? throw new Exception();\r\n\r\nstatic string GetString() => \"test\";<\/code><\/pre>\n
The JIT will be able to inline GetString<\/code>, at which point it can see that the result is non-null<\/code> and can fold away the check for the null<\/code> constant, at which point it can also dead-code eliminate the throw<\/code>. Constant folding is useful on its own in avoiding unnecessary work, but it also often unlocks other optimizations, like dead-code elimination and bounds-check elimination. The JIT is already quite good at finding constant folding opportunities, and gets better in .NET 10. Consider this benchmark:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\", \"s\")]\r\npublic partial class Tests\r\n{\r\n    [Benchmark]\r\n    [Arguments(\"test\")]\r\n    public ReadOnlySpan<char> Test(string s)\r\n    {\r\n        s ??= \"\";\r\n        return s.AsSpan();\r\n    }\r\n}<\/code><\/pre>\n
Here’s the assembly that gets produced for .NET 9:<\/p>\n
; .NET 9\r\n; Tests.Test(System.String)\r\n       push      rbp\r\n       mov       rbp,rsp\r\n       mov       rax,75B5D6200008\r\n       test      rsi,rsi\r\n       cmove     rsi,rax\r\n       test      rsi,rsi\r\n       jne       short M00_L01\r\n       xor       eax,eax\r\n       xor       edx,edx\r\nM00_L00:\r\n       pop       rbp\r\n       ret\r\nM00_L01:\r\n       lea       rax,[rsi+0C]\r\n       mov       edx,[rsi+8]\r\n       jmp       short M00_L00\r\n; Total bytes of code 41<\/code><\/pre>\n
Of particular note are those two test rsi,rsi<\/code> instructions, which are null<\/code> checks. The assembly starts by loading a value into rax<\/code>; that value is the address of the \"\"<\/code> string literal. It then uses test rsi,rsi<\/code> to check whether the s<\/code> parameter, which was passed into this instance method in the rsi<\/code> register, is null<\/code>. If it is null<\/code>, the cmove rsi,rax<\/code> instruction sets it to the address of the \"\"<\/code> literal. And then… it does test rsi,rsi<\/code> again? That second test is the null<\/code> check at the beginning of AsSpan<\/code>, which looks like this:<\/p>\n
public static ReadOnlySpan<char> AsSpan(this string? text)\r\n{\r\n    if (text is null) return default;\r\n    return new ReadOnlySpan<char>(ref text.GetRawStringData(), text.Length);\r\n}<\/code><\/pre>\n
Now with dotnet\/runtime#111985<\/a>, that second null<\/code> check, along with others, can be folded, resulting in this:<\/p>\n
; .NET 10\r\n; Tests.Test(System.String)\r\n       mov       rax,7C01C4600008\r\n       test      rsi,rsi\r\n       cmove     rsi,rax\r\n       lea       rax,[rsi+0C]\r\n       mov       edx,[rsi+8]\r\n       ret\r\n; Total bytes of code 25<\/code><\/pre>\n
Similar impact comes from dotnet\/runtime#108420<\/a>, which is also able to fold a different class of null<\/code> checks.<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\", \"condition\")]\r\npublic partial class Tests\r\n{\r\n    [Benchmark]\r\n    [Arguments(true)]\r\n    public bool Test(bool condition)\r\n    {\r\n        string tmp = condition ? GetString1() : GetString2();\r\n        return tmp is not null;\r\n    }\r\n\r\n    private static string GetString1() => \"Hello\";\r\n    private static string GetString2() => \"World\";\r\n}<\/code><\/pre>\n
In this benchmark, we<\/em> can see that neither GetString1<\/code> nor GetString2<\/code> return null<\/code>, and thus the is not null<\/code> check shouldn’t be necessary. The JIT in .NET 9 couldn’t see that, but its improved .NET 10 self can.<\/p>\n
; .NET 9\r\n; Tests.Test(Boolean)\r\n       mov       rax,7407F000A018\r\n       mov       rcx,7407F000A050\r\n       test      sil,sil\r\n       cmove     rax,rcx\r\n       test      rax,rax\r\n       setne     al\r\n       movzx     eax,al\r\n       ret\r\n; Total bytes of code 37\r\n\r\n; .NET 10\r\n; Tests.Test(Boolean)\r\n       mov       eax,1\r\n       ret\r\n; Total bytes of code 6<\/code><\/pre>\n
Constant folding also applies to SIMD (Single Instruction Multiple Data), instructions that enable processing multiple pieces of data at once rather than only one element at a time. dotnet\/runtime#117099<\/a> and dotnet\/runtime#117572<\/a> both enable more SIMD comparison operations to participate in folding.<\/p>\n
Code Layout<\/h3>\n
When the JIT compiler generates assembly from the IL emitted by the C# compiler, it organizes that code into “basic blocks,” a sequence of instructions with one entry point and one exit point, no jumps inside, no branches out except at the end. These blocks can then be moved around as a unit, and the order in which these blocks are placed in memory is referred to as “code layout” or “basic block layout.” This ordering can have a significant performance impact because modern CPUs rely heavily on an instruction cache and on branch prediction to keep things moving fast. If frequently executed (“hot”) blocks are close together and follow a common execution path, the CPU can execute them with fewer cache misses and fewer mispredicted jumps. If the layout is poor, where the hot code is split into pieces far apart from each other, or where rarely executed (“cold”) code sits in between, the CPU can spend more time jumping around and refilling caches than doing actual work. Consider a tight loop executed millions of times. A good layout keeps the loop entry, body, and backward edge (the jump back to the beginning of the body to do the next iteration) right next to each other, letting the CPU fetch them straight from the cache. In a bad layout, that loop might be interwoven with unrelated cold blocks (say, a catch<\/code> block for a try<\/code> in the loop), forcing the CPU to load instructions from different places and disrupting the flow. Similarly, for an if<\/code> block, the likely path should generally be the next block so no jump is required, with the unlikely branch behind a short jump away, as that better aligns with the sensibilities of branch predictors. Code layout heuristics control how that happens, and as a result, how efficient the resulting code is able to execute.<\/p>\n
When determining the starting layout of the blocks (before additional optimizations are done for the layout), dotnet\/runtime#108903<\/a> employs a “loop-aware reverse post-order” traversal. A reverse post-order traversal is an algorithm for visiting the nodes in a control flow graph such that each block appears after its predecessors. The “loop aware” part means the traversal recognizes loops as units, effectively creating a block around the whole loop, and tries to keep the whole loop together as the layout algorithm moves things around. The intent here is to start the larger layout optimizations from a more sensible place, reducing the amount of later reshuffling and situations where loop bodies get broken up.<\/p>\n
In the extreme, layout is essentially the traveling salesman problem<\/a>. The JIT must decide the order of basic blocks so that control transfers follow short, predictable paths and make efficient use of instruction cache and branch prediction. Just like the salesman visiting cities with minimal total travel distance, the compiler is trying to arrange blocks so that the “distance” between blocks, which might be measured in bytes or instruction fetch cost or something similar, is minimized. For any meaningfully-sized set of blocks, this is prohibitively expensive to compute optimally, as the number of possible orderings grows factorially with the number of blocks. Thus, the JIT has to rely on approximations rather than attempting an exact solution. One such approximation it employs now as of dotnet\/runtime#103450<\/a> (and then tweaked further in dotnet\/runtime#109741<\/a> and dotnet\/runtime#109835<\/a>) is a “3-opt,” which really just means that rather than considering all blocks together, it looks at only three and tries to produce an optimal ordering amongst those (there are only eight possible orderings to be checked). The JIT can choose to iterate through sets of three blocks until either it doesn’t see any more improvements or hits a self-imposed limit. Specifically when handling backward jumps, with dotnet\/runtime#110277<\/a>, it expands this “3-opt” to “4-opt” (four blocks).<\/p>\n
.NET 10 also does a better job of factoring PGO data into layout. With dynamic PGO, the JIT is able to gather instrumentation data from an initial compilation and then use the results of that profiling to impact an optimized re-compilation. That data can lead to conclusions about what blocks are hot or cold, and which direction branches take, all information that’s valuable for layout optimization. However, data can sometimes be missing from these profiles, so the JIT has a “profile synthesis” algorithm that makes realistic guesses for these gaps in order to fill them in (if you’ve read or seen “Jurassic Park,” this is the JIT-equivalent to filling in gaps in the dinosaur DNA sequences with that from present-day frogs.) With dotnet\/runtime#111915<\/a>, that repairing of the profile data is now performed just before layout, so that layout has a more complete picture.<\/p>\n
Let’s take a concrete example of all this. Here I’ve extracted the core function from MemoryExtensions.BinarySearch<\/code>:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\nusing System.Runtime.CompilerServices;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private int[] _values = Enumerable.Range(0, 512).ToArray();\r\n\r\n    [Benchmark]\r\n    public int BinarySearch()\r\n    {\r\n        int[] values = _values;\r\n        return BinarySearch(ref values[0], values.Length, 256);\r\n    }\r\n\r\n    [MethodImpl(MethodImplOptions.NoInlining)]\r\n    private static int BinarySearch<T, TComparable>(\r\n        ref T spanStart, int length, TComparable comparable)\r\n        where TComparable : IComparable<T>, allows ref struct\r\n    {\r\n        int lo = 0;\r\n        int hi = length - 1;\r\n        while (lo <= hi)\r\n        {\r\n            int i = (int)(((uint)hi + (uint)lo) >> 1);\r\n\r\n            int c = comparable.CompareTo(Unsafe.Add(ref spanStart, i));\r\n            if (c == 0)\r\n            {\r\n                return i;\r\n            }\r\n            else if (c > 0)\r\n            {\r\n                lo = i + 1;\r\n            }\r\n            else\r\n            {\r\n                hi = i - 1;\r\n            }\r\n        }\r\n\r\n        return ~lo;\r\n    }\r\n}<\/code><\/pre>\n
And here’s the assembly we get for .NET 9 and .NET 10, diff’d from the former to the latter:<\/p>\n
; Tests.BinarySearch[[System.Int32, System.Private.CoreLib],[System.Int32, System.Private.CoreLib]](Int32 ByRef, Int32, Int32)\r\n       push      rbp\r\n       mov       rbp,rsp\r\n       xor       ecx,ecx\r\n       dec       esi\r\n       js        short M01_L07\r\n+      jmp       short M01_L03\r\nM01_L00:\r\n-      lea       eax,[rsi+rcx]\r\n-      shr       eax,1\r\n-      movsxd    r8,eax\r\n-      mov       r8d,[rdi+r8*4]\r\n-      cmp       edx,r8d\r\n-      jge       short M01_L03\r\n       mov       r9d,0FFFFFFFF\r\nM01_L01:\r\n       test      r9d,r9d\r\n       je        short M01_L06\r\n       test      r9d,r9d\r\n       jg        short M01_L05\r\n       lea       esi,[rax-1]\r\nM01_L02:\r\n       cmp       ecx,esi\r\n-      jle       short M01_L00\r\n-      jmp       short M01_L07\r\n+      jg        short M01_L07\r\nM01_L03:\r\n+      lea       eax,[rsi+rcx]\r\n+      shr       eax,1\r\n+      movsxd    r8,eax\r\n+      mov       r8d,[rdi+r8*4]\r\n       cmp       edx,r8d\r\n-      jg        short M01_L04\r\n-      xor       r9d,r9d\r\n+      jl        short M01_L00\r\n+      cmp       edx,r8d\r\n+      jle       short M01_L04\r\n+      mov       r9d,1\r\n       jmp       short M01_L01\r\nM01_L04:\r\n-      mov       r9d,1\r\n+      xor       r9d,r9d\r\n       jmp       short M01_L01\r\nM01_L05:\r\n       lea       ecx,[rax+1]\r\n       jmp       short M01_L02\r\nM01_L06:\r\n       pop       rbp\r\n       ret\r\nM01_L07:\r\n       mov       eax,ecx\r\n       not       eax\r\n       pop       rbp\r\n       ret\r\n; Total bytes of code 83<\/code><\/pre>\n
We can see that the main change here is a block that’s moved (the bulk of M01_L00<\/code> moving down to M01_L03<\/code>). In .NET 9, the lo <= hi<\/code> “stay in the loop check” (cmp ecx,esi<\/code>) is a backward conditional branch (jle short M01_L00<\/code>), where every iteration of the loop except for the last jumps back to the top (M01_L00<\/code>). In .NET 10, it instead does a forward branch to exit the loop only in the rarer case, otherwise falling through to the body of the loop in the common case, and then unconditionally branching back.<\/p>\n
GC Write Barriers<\/h3>\n
The .NET garbage collector (GC) works on a generational model, organizing the managed heap according to how long objects have been alive. The newest allocations land in “generation 0” (gen0), objects that have survived at least one collection are promoted to “generation 1” (gen1), and those that have been around for longer end up in “generation 2” (gen2). This is based on the premise that most objects are temporary, and that once an object has been around for a while, it’s likely to stick around for a while longer. Splitting up the heap into generations enables for quickly collecting gen0 objects by only scanning the gen0 heap for remaining references to that object. The expectation is that all, or at least the vast majority, of references to a gen0 object are also in gen0. Of course, if a reference to a gen0 object snuck into gen1 or gen2, not scanning gen1 or gen2 during a gen0 collection could be, well, bad. To avoid that case, the JIT collaborates with the GC to track references from older to younger generations. Whenever there’s a reference write that could cross a generation, the JIT emits a call to a helper that tracks the information in a “card table,” and when the GC runs, it consults this table to see if it needs to scan a portion of the higher generations. That helper is referred to as a “GC write barrier.” Since a write barrier is potentially employed on every reference write, it must be super fast, and in fact the runtime has several different variations of write barriers so that the JIT can pick one optimized for the given situation. Of course, the fastest write barrier is one that doesn’t need to exist at all, so as with bounds checks, the JIT also exerts energy to try to prove when write barriers aren’t needed, eliding them when it can. And it can even more in .NET 10.<\/p>\n
ref structs<\/code>, referred to in runtime vernacular as “byref-like types,” can never live on the heap, which means any reference fields in them will similarly never live on the heap. As such, if the JIT can prove that a reference write is targeting a field of a ref struct<\/code>, it can elide the write barrier. Consider this example:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private object _object = new();\r\n\r\n    [Benchmark]\r\n    public MyRefStruct Test() => new MyRefStruct() { Obj1 = _object, Obj2 = _object, Obj3 = _object };\r\n\r\n    public ref struct MyRefStruct\r\n    {\r\n        public object Obj1;\r\n        public object Obj2;\r\n        public object Obj3;\r\n    }\r\n}<\/code><\/pre>\n
In the .NET 9 assembly, we can see three write barriers (CORINFO_HELP_CHECKED_ASSIGN_REF<\/code>) corresponding to the three fields in MyRefStruct<\/code> in the benchmark:<\/p>\n
; .NET 9\r\n; Tests.Test()\r\n       push      r15\r\n       push      r14\r\n       push      rbx\r\n       mov       rbx,rsi\r\n       mov       r15,[rdi+8]\r\n       mov       rsi,r15\r\n       mov       r14,r15\r\n       mov       rdi,rbx\r\n       call      CORINFO_HELP_CHECKED_ASSIGN_REF\r\n       lea       rdi,[rbx+8]\r\n       mov       rsi,r14\r\n       call      CORINFO_HELP_CHECKED_ASSIGN_REF\r\n       lea       rdi,[rbx+10]\r\n       mov       rsi,r15\r\n       call      CORINFO_HELP_CHECKED_ASSIGN_REF\r\n       mov       rax,rbx\r\n       pop       rbx\r\n       pop       r14\r\n       pop       r15\r\n       ret\r\n; Total bytes of code 59<\/code><\/pre>\n
With dotnet\/runtime#111576<\/a> and dotnet\/runtime#111733<\/a> in .NET 10, all of those write barriers are elided:<\/p>\n
; .NET 10\r\n; Tests.Test()\r\n       mov       rax,[rdi+8]\r\n       mov       rcx,rax\r\n       mov       rdx,rax\r\n       mov       [rsi],rcx\r\n       mov       [rsi+8],rdx\r\n       mov       [rsi+10],rax\r\n       mov       rax,rsi\r\n       ret\r\n; Total bytes of code 25<\/code><\/pre>\n
Much more impactful, however, are dotnet\/runtime#112060<\/a> and dotnet\/runtime#112227<\/a>, which have to do with “return buffers.” When a .NET method is typed to return a value, the runtime has to decide how that value gets from the callee back to the caller. For small types, like integers, floating-point numbers, pointers, or object references, the answer is simple: the value can be passed back via one or more CPU registers reserved for return values, making the operation essentially free. But not all values fit neatly into registers. Larger value types, such as structs with multiple fields, require a different strategy. In these cases, the caller allocates a “return buffer,” a block of memory, typically in the caller’s stack frame, and the caller passes a pointer to that buffer as a hidden argument to the method. The method then writes the return value directly into that buffer in order to provide the caller with the data. When it comes to write barriers, the challenge here is that there historically hasn’t been a requirement that the return buffer be on the stack; it’s technically feasible it could have been allocated on the heap, even if it rarely or never is. And since the callee doesn’t know where the buffer lives, any object reference writes needed to be tracked with GC write barriers. We can see that with a simple benchmark:<\/p>\n
\/\/ dotnet run -c Release -f net9.0 --filter \"*\" --runtimes net9.0 net10.0\r\n\r\nusing BenchmarkDotNet.Attributes;\r\nusing BenchmarkDotNet.Running;\r\n\r\nBenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);\r\n\r\n[DisassemblyDiagnoser]\r\n[HideColumns(\"Job\", \"Error\", \"StdDev\", \"Median\", \"RatioSD\")]\r\npublic partial class Tests\r\n{\r\n    private string _firstName = \"Jane\", _lastName = \"Smith\", _address = \"123 Main St\", _city = \"Anytown\";\r\n\r\n    [Benchmark]\r\n    public Person GetPerson() => new(_firstName, _lastName, _address, _city);\r\n\r\n    public record struct Person(string FirstName, string LastName, string Address, string City);\r\n}<\/code><\/pre>\n
On .NET 9, each field of the returned value type is incurring a CORINFO_HELP_CHECKED_ASSIGN_REF<\/code> write barrier:<\/p>\n
; .NET 9\r\n; Tests.GetPerson()\r\n       push      r15\r\n       push      r14\r\n       push      r13\r\n       push      rbx\r\n       mov       rbx,rsi\r\n       mov       rsi,[rdi+8]\r\n       mov       r15,[rdi+10]\r\n       mov       r14,[rdi+18]\r\n       mov       r13,[rdi+20]\r\n       mov       rdi,rbx\r\n       call      CORINFO_HELP_CHECKED_ASSIGN_REF\r\n       lea       rdi,[rbx+8]\r\n       mov       rsi,r15\r\n       call      CORINFO_HELP_CHECKED_ASSIGN_REF\r\n       lea       rdi,[rbx+10]\r\n       mov       rsi,r14\r\n       call      CORINFO_HELP_CHECKED_ASSIGN_REF\r\n       lea       rdi,[rbx+18]\r\n       mov       rsi,r13\r\n       call      CORINFO_HELP_CHECKED_ASSIGN_REF\r\n       mov       rax,rbx\r\n       pop       rbx\r\n       pop       r13\r\n       pop       r14\r\n       pop       r15\r\n       ret\r\n; Total bytes of code 81<\/code><\/pre>\n
Now in .NET 10, the calling convention has been updated to require that the return buffer live on the stack (if the caller wants the data somewhere else, it’s responsible for subsequently doing that copy). And because the return buffer is now guaranteed to be on the stack, the JIT can elide all GC write barriers as part of returning values.<\/p>\n
; .NET 10\r\n; Tests.GetPerson()\r\n       mov       rax,[rdi+8]\r\n       mov       rcx,[rdi+10]\r\n       mov       rdx,[rdi+18]\r\n       mov       rdi,[rdi+20]\r\n       mov       [rsi],rax\r\n       mov       [rsi+8],rcx\r\n       mov       [rsi+10],rdx\r\n       mov       [rsi+18],rdi\r\n       mov       rax,rsi\r\n       ret\r\n; Total bytes of code 35<\/code><\/pre>\n
dotnet\/runtime#111636<\/a> from @a74nh<\/a> is also interesting from a performance perspective because, as is common in optimization, it trades off one thing for another. Prior to this change, Arm64 had one universal write barrier helper for all GC modes. This change brings Arm64 in line with x64 by routing through a WriteBarrierManager<\/code> that selects among multiple JIT_WriteBarrier<\/code> variants based on runtime configuration. In doing so, it makes each Arm64 write barrier a bit more expensive, by adding region checks and moving to a region-aware card marking scheme, but in exchange it lets the GC do less work: fewer cards in the card table get marked, and the GC can scan more precisely. dotnet\/runtime#106191<\/a> also helps reduce the cost of write barriers on Arm64 by tightening the hot-path comparisons and eliminating some avoidable saves and restores.<\/p>\n
Instruction Sets<\/h3>\n
.NET continues to see meaningful optimizations and improvements across all supported architectures, along with various architecture-specific improvements. Here are a handful of examples.<\/p>\n
Arm SVE<\/h4>\n
APIs for Arm SVE were introduced in .NET 9. As noted in the Arm SVE<\/a> section of last year’s post, enabling SVE is a multi-year effort, and in .NET 10, support is still considered experimental. However, the support has continued to be improved and extended, with PRs like dotnet\/runtime#115775<\/a> from @snickolls-arm<\/a> adding BitwiseSelect<\/code> methods, dotnet\/runtime#117711<\/a> from @jacob-crawley<\/a> adding MaxPairwise<\/code> and MinPairwise<\/code> methods, and dotnet\/runtime#117051<\/a> from @jonathandavies-arm<\/a> adding VectorTableLookup<\/code> methods.<\/p>\n
Arm64<\/h4>\n
dotnet\/runtime#111893<\/a> from @jonathandavies-arm<\/a>, dotnet\/runtime#111904<\/a> from @jonathandavies-arm<\/a>, dotnet\/runtime#111452<\/a> from @jonathandavies-arm<\/a>, dotnet\/runtime#112235<\/a> from @jonathandavies-arm<\/a>, and dotnet\/runtime#111797<\/a> from @snickolls-arm<\/a> all improved .NET’s support for utilizing Arm64’s multi-operation compound instructions. For example, when implementing a compare and branch, rather than emitting a cmp<\/code> against 0 followed by beq<\/code> instruction, the JIT may now emit a cbz<\/code> (“Compare and Branch on Zero”) instruction.<\/p>\n
APX<\/h4>\n
Intel’s Advanced Performance Extensions (APX) was announced in 2023 as an extension of the x86\/x64 instruction set. It expands the number of general-purpose registers from 16 to 32 and adds new instructions such as conditional operations designed to reduce memory traffic, improve performance, and lower power consumption. dotnet\/runtime#106557<\/a> from @Ruihan-Yin<\/a>, dotnet\/runtime#108796<\/a> from @Ruihan-Yin<\/a>, and dotnet\/runtime#113237<\/a> from @Ruihan-Yin<\/a> essentially teach the JIT how to speak the new dialect of assembly code (the REX and expanded EVEX encodings), while dotnet\/runtime#108799<\/a> from @DeepakRajendrakumaran<\/a> updates the JIT to be able to use the expanded set of registers, and dotnet\/runtime#116035<\/a> from @DeepakRajendrakumaran<\/a> enables new push and pop instructions for working with such registers. The most impactful new instructions in APX are around conditional compares (ccmp<\/code>), a concept the JIT already supports from targeting other instruction sets, and dotnet\/runtime#111072<\/a> from @anthonycanino<\/a>, dotnet\/runtime#112153<\/a> from @anthonycanino<\/a>, and dotnet\/runtime#116445<\/a> from @khushal1996<\/a> all teach the JIT how to make good use of these new instructions with APX.<\/p>\n
AVX512<\/h4>\n
.NET 8 added broad support for AVX512, and .NET 9 significantly improved its handling and adoption throughout the core libraries. .NET 10 includes a plethora of additional related optimizations:<\/p>\n