Copy-on-Write in Win32 API Early Access
(Updated Apr 4 and 26, 2024 with some release news.)
On October 25, 2023, the Windows filesystem team released an early preview of copy-on-write (CoW) linking in the Windows 11 Insider Canary channel. This builds automatic CoW linking into the Win32 CopyFile APIs when using Dev Drive or ReFS. If released next year, this will eliminate the need to update build engines, tools, and runtime frameworks to support CoW. Related release information is here.
We released some early benchmarks showing the automatic gain for .NET File.Copy() without the need for explicitly updating tools to use CoW. In testing a large C# repo under this new configuration, we saw a 32% reduction in build time just moving to Dev Drive, then CoW linking increased that win to 42%.
See our previous CoW and Dev Drive coverage here and here.
If you have builds or other workloads that do a lot of copying, join a machine to the Canary Insider release and run benchmarks on a Dev Drive to discover the gains you can achieve.
CoW-in-Win32 will ship as part of Windows Server 2025 as well as in Windows 11 in the second half of 2024. Dev Drive was not backported to Windows Server 2022. Preview versions of both operating systems have CoW-in-Win32 turned on by default, so you can perform early testing.
Light
Dark
5 comments
I’ve noticed that none of the Dev Drive blogs seem to talk about c++ project build times which is quite surprising given the nature of that language.
Is there anything coming to showcase wins there?
I imagine that baking CoW into the win32 API will allow c++ scenarios to benefit beyond just hosting them on Dev Drive correct?
I’ve been testing several large MSBuild MSVC repos. There is a win there, often dependent on how much C++ there is vs. C# or other workloads. MSVC tools are computationally heavy vs. C#. Also, .vcxproj projects do not have the assembly copy snowball problem that C# has, so CoW doesn’t help much unless the user-defined project code copies MSVC outputs to other locations on-disk.
For example, one repo with 1/3 C# and 2/3 C++ sees an overall 2% win from the move to Dev Drive – saving 20-40 seconds vs. the NTFS baseline – but the overall build time of 2000 seconds means this is only a 1-2% win. During the build a great number of cl.exe processes are running and maxing out the box; the C# shared compilation process contributes only lightly to the overall load.
Another repo with about 2/3 C# and 1/3 C++ with a lot of copying to assemble microservice layouts sees a 13% win from DD and CoW. Not bad, just not the 43% seen by one of the large pure-C# repos.
I forgot a couple of additional things to try on your machine:
* MSVC makes extensive use of %TEMP% to write temp files. Measure with moving TEMP and TMP to point to a location on the Dev Drive. I’ve tried this a couple of times and any benefit was in the noise, but maybe it will show a clear signal for you.
* I see a 3-5% improvement in build times generally – typically on the higher side for C++ repos – by turning on the “Disable Write Cache Flushing” checkbox in the Device Manager properties for your machine’s disk (or the disk where build is happening). Given the warning text in the Windows GUI it’s probably more ideal to use on a laptop or a datacenter VM, but given the number of tiny temp and status files written by MSBuild and MSVC, this switch means more overlapping of disk cache writes with later processes, as the write cache has more time to flush lazily.
So… this would mean that Windows Explorer and Robocopy would be accelerated too? Even maybe git operations? (Although I notice git-lfs already tries to use “FSCTL_DUPLICATE_EXTENTS_TO_FILE” when it can.)
I was getting disappointed that .Net Core on Linux and Mac has CoW-style copies built into File.Copy (Windows wasn’t!) but of course Kernel support would achieve this – and more! Yay!
For low-risk developer computers like me, I’d want to turn the whole C:\ into a ‘dev drive’! (Moving apps like Azure Data Studio to DevDrive feels faster to start-up, although I haven’t done any objective testing.)
Yes this would fix all copy code paths that use the Win32 API, regardless of the tool, so robocopy, xcopy, cmd.exe “copy” command, non-.NET languages, and so on. There was a PR for .NET 8/9 that was abandoned because of this support upcoming.
C: might be possible to make into a ReFS in the future, but even if so Dev Drive won’t be possible because a bunch of the speedup comes from strictly limiting the filesystem filter drivers that get to participate – the more filters the more callbacks for every single I/O call – and C: by definition needs all of the filters to apply to it. We’ll see where the filesystem ecosystem goes over the coming Windows releases.