Brian Dellisanti asked why ReadDirectoryChangesW didn’t provide a way to correlate the two sides of a rename operation.
I wasn’t there, but I can guess.
My guess is that the implementation always generated the two events one right after the other, so “obviously” the way you correlate them is to save the old name when you see the FILE_, and when the FILE_ comes immediately after, you have your two sides.
But they never wrote down that the two events always occur in direct succession. Which meant that when new file systems came along, they might not honor the unwritten rule. If two files are being renamed at the same time, is it possible that the two sets of rename events end up interleaved? There was nothing written down to forbid it, so I guess it’s possible.
Note that I don’t know whether any file systems actually break this unwritten rule. From what I can tell, they do generate the two events in rapid succession, but rapid succession doesn’t a priori guarantee that they will come directly one after the other, particularly if there is a lot of concurrent disk activity going on.
In practice, I couldn’t find a lot of code tracking renames anyway. They generally treated the FILE_ as a deletion and the FILE_ as a creation. And the ones that did track renames assumed that renames did not interleave. (Not that they had much choice.)
I don’t think that providing the file IDs for the two sides of a rename operation was the purpose of ReadDirectoryChangesExW‘s ReadDirectoryNotifyExtendedInformation. It was just a happy side effect that the extra information in the ReadDirectoryNotifyExtendedInformation also gives you the pieces needed to connect the dots reliably.
I thought you might appreciate me pointing out the trick, that’s all.
This blog always leaves me questioning assumptions I’ve held for decades.. is the file system a fundamentally single-threaded thing? If I had a dozen threads renaming files.. say, on a device with write-behind caching enabled. Would the notification events still be serialized? (Is that behavior file-system dependent?)
I think this potentially comes down to an implementation detail. They must be thread-safe, and probably parallelise easy operations. But consider the storage layer below the filesystem. It could be an NVMe drive, with support for thousands of parallel operations in hardware queues. Or it could be an IDE drive with support for one operation at a time, that has to physically seek on a disk.
With filesystems, it could be something like Ext2 with separate inodes that could be operated upon independently, or something like FAT where there's a volume-global cluster list, and parallel operations seem like a recipe for...
If you look inside the Linux kernel you will find the filesystem is *not* single threaded; but operations that create file names lock the directory first. So each directory is single threaded, mostly.
In theory a BTree or HTree filesystem could exist that violates this assumption and be made to still work; but I’ve not seen such a beast.
I think this may depend on what operations the notification queue supports. If the queue supports atomically pushing multiple consecutive messages, then interleaving will not occur. Alternatively, if the queue supports locking, two messages can be sent within a critical section and then unlocked. Of course, some operations on the filesystem may be serialized, for example, modifying metadata, and this is also possible. Sending messages during these serialized processes also does not involve interleaving.
The only trhing I am really interested in is why ReadDirectoryChanges and friends never bothered to provide file close notifications.
Because closing a file doesn’t change the directory.
If ReadDirectoryChangesExW were guaranteed to emit these two messages consecutively without interleaving, life would be easier.
ooook