On using milliseconds as a measure of network latency
One of the things I do is serve as an API design reviewer, reviewing and providing feedback on all new APIs added to Windows. There was a network property being added that reported the latency of a network connection. One of the other API reviewers put a note on that property asking, “As network technology improves, will millisecond granularity for reporting latency be sufficient, or should we use microseconds or even nanoseconds?”
I was not the team responsible for the new property, but I felt compelled to clarify the situation: “The speed of light is unlikely to improve.”
(I mean, maybe some scientific breakthrough could result in improvements, but that would result in a much more fundamental change to the design of the new property: It would have to be changed from an unsigned to a signed type!)
Bonus chatter: Now, even with current technology, a smaller unit could still be useful, if the other system is so close by that the latency is less than one millisecond. But once you get that close, the variability in latency begins to overwhelm the actual value. And the target audience for this particular network property was multiplayer online games, so anything less than one millisecond is “amazingly great”. You’d be thrilled to have the problem of having to choose among multiple sub-millisecond game partners!
Light
Dark
11 comments
I always thought that “social skills of a thermonuclear device” should have been it’s own category here.
I can imagine a data center of the future where the entire data center lives on single chip, and all the network communications are virtualised such that they appear to be communicating over ethernet, but internally it’s happening at the speed of the CPU. So nanosecond latency might not be as wild as it sounds.
Of course, once you exit the data center, you’d be back to normal speeds, but the servers within the data center communicating with each other might need metrics.
Déjà vu
I thought this post looked familiar!
“640K ought to be enough for anybody.”
FWIW, that’s probably apocryphal, but when I’ve gone digging, the people who can place it in any form of context have Mr Gates saying that 640K ought to be enough for anybody using MS-DOS, and that you should switch to Microsoft Xenix if MS-DOS is too restrictive.
Similar applies here – the context is multiplayer games, where everything else that affects play (such as input lag, frame output delay etc) is also measured in milliseconds. If we were talking about an interface for debugging datacentre networks, or on-chip data movements, milliseconds would be an unreasonable data unit, but we’re not.
It can also be fun when people are worrying about this to turn time into distance limit; 3 microseconds is approximately 1 kilometer at the speed of light and 1 millisecond is approximately 300 km. Is it important in this interface to be able to distinguish “peer must be within walking distance (microseconds)” from “peer must be in the same country (milliseconds)”? Or, for the nanosecond suggestion, “peer must be in touching distance”, since 1 nanosecond is around 0.3 meters?
Sometimes, this degree of resolution is actually useful; if you’re looking at being able to distinguish “same box” from “same rack” from “same hall in the datacenter” from “another datacenter”, you can need nanosecond resolution. But mostly, we’re not trying to do that, so don’t need that much accuracy.
Electrons in chips and on circuit boards don’t travel at the speed of light. They travel slower. About half the speed of light on FR4 circuit board.
Electrons don’t travel at half the speed of light on FR4; they travel at around 5 to 10 cm per hour. The electromagnetic wave travels at about half the speed of light on an FR4 circuit board. Similar applies in optical fibre – the electromagnetic wave of interest travels at around 2/3rds the speed of light. For most cables intended to carry communication signals, you can find the “velocity factor” listed, which is the speed of the electromagnetic wave traveling through that cable; for coax, this can be as low as 66% c, and as high as 93% c, while ladder line can be as high as 99% and plenum twisted pair can go as low as 40% c.
But if someone’s worried about “is milliseconds a reasonable unit?”, having the rule of thumb to hand is useful – 1 to 3 microseconds for a kilometer, 1 to 3 nanoseconds for a meter, 1 to 3 milliseconds for 1,000 km. It’s not a precise answer, it’s a guideline for whether the request makes sense. If distances in meters are too small to make sense, then nanoseconds are too small for latency. If distances in kilometers are too small, then microseconds are too small. And if the smallest sensible unit is 1,000 kilometers, then milliseconds are just right.
“will millisecond granularity for reporting latency be sufficient, or should we use microseconds or even nanoseconds”
Several posts suggesting that it depends on distance, which I would counter by saying that the granularity has nothing to do with the distance.
If the granularity is only millisecond, then one will not see changes in the latency of under one millisecond.
Microsoft operating systems can measure latency granularity in the nanosecond range with PTP, the total latency of the connection is irrelevant.
It took me decades to break free of the shackles of, “Don’t use floats because floating point math is slow. Use integer math.”
It wasn’t until Eric Brumer’s talks (e.g. “Native Code Performance and Memory – The Elephant in the CPU“) where he pointed out that a modern CPU can take the square root of a 32-bit number in the amount of time it takes to get a value out of the L2 cache and into a place where it can be used.
Performing computations is no longer what the CPU spends most of it’s time doing. Only about 1% of the silicon, and approximiately 0% of the CPU time, is dedicated to computing. The rest is waiting on memory, re-writing your machine code so it can fetch values before your code needed them, running instructions in a different order.
So, yes, it really is OK to use floats, even in performance-criticial timing code.
So, make it:
And the smallest timing tick that Windows can give you is 100 ns, meaning you could measure latency with a resolution down to:
Bonus Reading: We can’t e-mail over 500 miles
postscript: the modern x86 CPU JITTing your machine code on-the-fly reminds me of the promise of Itanium: it will be faster – we just have to invent the smart enough compiler:
> The Itanium is a 64-bit EPIC architecture. EPIC stands for Explicitly Parallel Instruction Computing, a design in which work is offloaded from the processor to the compiler. For example, the compiler decides which operations can be safely performed in parallel and which memory fetches can be productively speculated. This relieves the processor from having to make these decisions on the fly, thereby allowing it to focus on the real work of processing.
Which never really worked out, because writing the smart enough compiler is hard.
So instead Intel put the compiler inside the CPU, and translates your CISC x86 into it’s internal RISC micro-ops.