gRPC is a modern, cross-platform, high-performance RPC framework. gRPC for .NET is built on top of ASP.NET Core and is our recommended way to build RPC services using .NET.
.NET 6 further improves gRPC’s already great performance and adds a new range of features that make gRPC better than ever in modern cloud-native apps. In this post I’ll describe these new features as well as how we are leading the industry with the first gRPC implementation to support end-to-end HTTP/3.
gRPC client-side load balancing
Client-side load balancing is a feature that allows gRPC clients to distribute load optimally across available servers. Client-side load balancing can eliminate the need to have a proxy for load balancing. This has several benefits:
- Improved performance. No proxy means eliminating an additional network hop and reduced latency because RPCs are sent directly to the gRPC server.
- Efficient use of server resources. A load-balancing proxy must parse and then resend every HTTP request sent through it. Removing the proxy saves CPU and memory resources.
- Simpler application architecture. Proxy server must be set up and configured correctly. No proxy server means fewer moving parts!
Client-side load balancing is configured when a channel is created. The two components to consider when using load balancing:
- The resolver, which resolves the addresses for the channel. Resolvers support getting addresses from an external source. This is also known as service discovery.
- The load balancer, which creates connections and picks the address that a gRPC call will use.
The following code example configures a channel to use DNS service discovery with round-robin load balancing:
var channel = GrpcChannel.ForAddress(
"dns:///my-example-host",
new GrpcChannelOptions
{
Credentials = ChannelCredentials.Insecure,
ServiceConfig = new ServiceConfig { LoadBalancingConfigs = { new RoundRobinConfig() } }
});
var client = new Greet.GreeterClient(channel);
var response = await client.SayHelloAsync(new HelloRequest { Name = "world" });
For more information, see gRPC client-side load balancing.
Transient fault handling with retries
gRPC calls can be interrupted by transient faults. Transient faults include:
- Momentary loss of network connectivity.
- Temporary unavailability of a service.
- Timeouts due to server load.
When a gRPC call is interrupted, the client throws an RpcException with details about the error. The client app must catch the exception and choose how to handle the error.
var client = new Greeter.GreeterClient(channel);
try
{
var response = await client.SayHelloAsync(
new HelloRequest { Name = ".NET" });
Console.WriteLine("From server: " + response.Message);
}
catch (RpcException ex)
{
// Write logic to inspect the error and retry
// if the error is from a transient fault.
}Duplicating retry logic throughout an app is verbose and error-prone. Fortunately, the .NET gRPC client now has built-in support for automatic retries. Retries are centrally configured on a channel, and there are many options for customizing retry behavior using a RetryPolicy.
var defaultMethodConfig = new MethodConfig
{
Names = { MethodName.Default },
RetryPolicy = new RetryPolicy
{
MaxAttempts = 5,
InitialBackoff = TimeSpan.FromSeconds(1),
MaxBackoff = TimeSpan.FromSeconds(5),
BackoffMultiplier = 1.5,
RetryableStatusCodes = { StatusCode.Unavailable }
}
};
// Clients created with this channel will automatically retry failed calls.
var channel = GrpcChannel.ForAddress("https://localhost:5001", new GrpcChannelOptions
{
ServiceConfig = new ServiceConfig { MethodConfigs = { defaultMethodConfig } }
});For more information, see Transient fault handling with gRPC retries.
Protobuf performance
gRPC for .NET uses the Google.Protobuf package as the default serializer for messages. Protobuf is an efficient binary serialization format. Google.Protobuf is designed for performance, using code generation instead of reflection to serialize .NET objects. In .NET 5 we worked with the Protobuf team to add support for modern memory APIs such as Span<T>, ReadOnlySequence<T>, and IBufferWriter<T> to the serializer. The improvements in .NET 6 optimize an already fast serializer.
protocolbuffers/protobuf#8147 adds vectorized string serialization. SIMD instructions allow multiple characters to be processed in parallel, dramatically increasing performance when serializing certain string values.
private string _value = new string(' ', 10080);
private byte[] _outputBuffer = new byte[10080];
[Benchmark]
public void WriteString()
{
var span = new Span<byte>(_outputBuffer);
WriteContext.Initialize(ref span, out WriteContext ctx);
ctx.WriteString(_value);
ctx.Flush();
}| Method | Google.Protobuf | Mean | Ratio | Allocated |
|---|---|---|---|---|
| WriteString | 3.14 | 8.838 us | 1.00 | 0 B |
| WriteString | 3.18 | 2.919 ns | 0.33 | 0 B |
protocolbuffers/protobuf#7645 adds a new API for creating ByteString instances. If you know the underlying data won’t change, then use UnsafeByteOperations.UnsafeWrap to create a ByteString without copying the underlying data. This is very useful if an app works with large byte payloads and you want to reduce garbage collection frequency.
var data = await File.ReadAllBytesAsync(@"c:large_file.json");
// Safe but slow.
var copied = ByteString.CopyFrom(data);
// Unsafe but fast. Useful if you know data won't change.
var wrapped = UnsafeByteOperations.UnsafeWrap(data);gRPC download speeds
gRPC users reported sometimes getting slow download speeds. Our investigation discovered that HTTP/2 flow control was constraining downloads when there is latency between the client and server. The server fills the receive buffer window before the client can drain it, causing the server to pause sending data. gRPC messages are downloaded in start/stop bursts.
This is fixed in dotnet/runtime#54755. HttpClient now dynamically scales the receive buffer window. When an HTTP/2 connection is established, the client will send a ping to the server to measure latency. If there is high latency, the client automatically increases the receive buffer window, enabling a fast, continuous download.
private GrpcChannel _channel = GrpcChannel.ForAddress(...);
private DownloadClient _client = new DownloadClient(_channel);
[Benchmark]
public Task GrpcLargeDownload() =>
_client.DownloadLargeMessageAsync(new EmptyMessage());| Method | Runtime | Mean | Ratio |
|---|---|---|---|
| GrpcLargeDownload | .NET 5.0 | 6.33 s | 1.00 |
| GrpcLargeDownload | .NET 6.0 | 1.65 s | 0.26 |
HTTP/3 support
gRPC on .NET now supports HTTP/3. gRPC builds on top of HTTP/3 support added to ASP.NET Core and HttpClient in .NET 6. For more information, see HTTP/3 support in .NET 6.
.NET is the first gRPC implementation to support end-to-end HTTP/3, and we have submitted a gRFC for other platforms to support HTTP/3 in the future. gRPC with HTTP/3 is a highly requested feature by the developer community, and it is exciting to see .NET leading the way in this area.
Wrapping Up
Performance is a feature of .NET and gRPC, and .NET 6 is faster than ever. New performance-orientated features like client-side load balancing and HTTP/3 mean lower latency, higher throughput, and fewer servers. It is an opportunity to save money, reduce power use and build greener cloud-native apps.
To try out the new features and get started using gRPC with .NET, the best place to start is the Create a gRPC client and server in ASP.NET Core tutorial.
We look forward to hearing about apps built with gRPC and .NET and to your future contributions in the dotnet and grpc repos!
I am trying to use Milkman to smoke test my gRPC app. It has an option to "use reflection" to discover the protobuf. But my app apparently does not support reflection. In Googling how to enable it, all the results I find point to older protobuf versions and/or tools. Is there support for reflection in the the .net 6 protobuf world. (It would be nice if it were a feature of the protobug compiler.)
This is great set of improvements. One question though, given that client side load balancing also makes a network call to determine server endpoint, how does it save time compared against a load balancer?
The client resolves server endpoints and caches them when it first starts. Additional gRPC calls use the cache.
The No Proxy and Performance sounds cool (in the context of gRPC vs Message Broker), but what about browser clients, persisted buffering and broadcasting.
Is this “.NET build in” or on the roadmap somehow?
Hi James
I’m looking forward to use this in my org, but it seems to still be in preview (according to the doc). Is there an ETA for the official release?
Keep up the great work!
Hopefully in the next couple of months.
It’s sad that this new gRPC library doesn’t work on .NET Framework (‘classic’) – for those that can’t modify the projects to work with .NET Core in the near future, we are stuck with the old gRpc library, that will no longer receive bugfixes or improvements as far as I know.. 🙁
The new client – Grpc.Net.Client – has some support for .NET Framework. See https://docs.microsoft.com/aspnet/core/grpc/supported-platforms#net-grpc-client-requirements
Thanks - unfortunately we have a lot of code also on the client-side on mono 6 under Linux (RedHat, Ubuntu etc..) that is using gRpc to connect to a server. It's not very clear from that page if and how mono 6 on Linux is supported by grpc-dotnet, when connecting to a gRpc server implemented using the 'old' gRPC Core C# library. (the server is not an ASP.NET application also).
Our configuration is:
Client apps:...
Hi James,
Can you please elaborate about next steps for .Net Framework (“classic”). What is the road map? Any chance that situation will be improved in it will support more gRPC features? Our Legacy code is 4.8 and we not going move to .NET Core (only brand new projects). So we need better .NET Framework Client connectivity for gRPC.
10x in advance
Hi James
On the recommendation of a friend, I wrote my first gRPC dotnet client/server app. It is blisteringly fast compared to the old WCF version. I added the transient fault handling with retries code to my client, as per your example. When I run the client with the server off, deadline handling appears to trump the retries (I think - not sure). Are deadlines and retries mutually exclusive? If not,...
You’re correct. A deadline trumps retries. Once the deadline is exceeded then the in-progress request is canceled and no more retries are attempted.
This is a good thing to point out in the documentation. I will add it.
https://github.com/dotnet/AspNetCore.Docs/pull/24218
This doc does not seem to reflect those latest changes: https://docs.microsoft.com/en-us/aspnet/core/grpc/clientfactory?view=aspnetcore-6.0
Could you go into more detail about what changes you want to see in docs?
hey @James is it possible to share the code for the client?? the one you used on the video on the dotnet conf.. I would like to know how do you do to update all the server status on the screen. thx.
It’s available here: https://github.com/grpc/grpc-dotnet/tree/master/examples#container
Please take care that gRPC Server will be available on Android and iOS .Net 6 aka MAUI.
The deprecation of the native gRPC Server leaves a big hole on these platforms.
https://github.com/dotnet/aspnetcore/issues/35077
I don’t know if I’m missing something but in the protobuf version speed comparison table it says 2.919 ns is only 1/3rd of 8.838 us.
8.838 X 0.33 = 2.91654, so 1/3rd of 8.838, not 1/3rd less
Thanks, I fixed the wording. This is still confusing though if you note that the units are off by 1000.