{"id":49135,"date":"2023-11-28T10:05:00","date_gmt":"2023-11-28T18:05:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/dotnet\/?p=49135"},"modified":"2024-12-13T14:03:14","modified_gmt":"2024-12-13T22:03:14","slug":"building-resilient-cloud-services-with-dotnet-8","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/building-resilient-cloud-services-with-dotnet-8\/","title":{"rendered":"Building resilient cloud services with .NET 8"},"content":{"rendered":"<p>Building resilient apps is a fundamental requirement for cloud development. With .NET 8, we&#8217;ve made substantial advancements to simplify the integration of resilience into your applications. We&#8217;re excited to introduce the <a href=\"https:\/\/www.nuget.org\/packages\/Microsoft.Extensions.Http.Resilience\"><code>Microsoft.Extensions.Http.Resilience<\/code><\/a> and <a href=\"https:\/\/www.nuget.org\/packages\/Microsoft.Extensions.Resilience\"><code>Microsoft.Extensions.Resilience<\/code><\/a> packages to the .NET ecosystem. These new libraries are based on the <a href=\"https:\/\/github.com\/App-vNext\/Polly\">Polly<\/a> library, a widely recognized open-source project.<\/p>\n<h2>TL;DR<\/h2>\n<p>To use the new HTTP resilience APIs, install the package from the command line:<\/p>\n<pre><code class=\"language-dotnetcli\">dotnet add package Microsoft.Extensions.Http.Resilience<\/code><\/pre>\n<p>Or add it directly in the C# project file:<\/p>\n<pre><code class=\"language-xml\">&lt;ItemGroup&gt;\r\n  &lt;PackageReference Include=\"Microsoft.Extensions.Http.Resilience\" \/&gt;\r\n&lt;\/ItemGroup&gt;<\/code><\/pre>\n<p>You can now use the recommended <code>AddStandardResilienceHandler<\/code> extension on the <code>IHttpClientBuilder<\/code>:<\/p>\n<pre><code class=\"language-csharp\">var services = new ServiceCollection();\r\n\r\nservices.AddHttpClient(\"my-client\")\r\n        .AddStandardResilienceHandler(options =&gt;\r\n        {\r\n            \/\/ Configure standard resilience options here\r\n        });<\/code><\/pre>\n<p>Example above uses <code>AddStandardResilienceHandler<\/code> to add a pipeline (rate limiter, total timeout, retry, circuit breaker, attempt timeout) of resilience strategies to the HTTP client. See <a href=\"#standard-resilience-pipeline\">standard resilience pipeline<\/a> section to learn more.<\/p>\n<p>A more real-world example would rely on hosting, such as that described in the <a href=\"https:\/\/learn.microsoft.com\/dotnet\/core\/extensions\/generic-host\">.NET Generic Host<\/a> article. Using the <a href=\"https:\/\/www.nuget.org\/packages\/Microsoft.Extensions.Hosting\">Microsoft.Extensions.Hosting<\/a> NuGet package, the above example becomes:<\/p>\n<pre><code class=\"language-csharp\">using Http.Resilience.Example;\r\nusing Microsoft.Extensions.DependencyInjection;\r\nusing Microsoft.Extensions.Hosting;\r\n\r\nHostApplicationBuilder builder = Host.CreateApplicationBuilder(args);\r\nIServiceCollection services = builder.Services;\r\n\r\nservices.AddHttpClient(\"my-client\")\r\n        .AddStandardResilienceHandler(options =&gt;\r\n        {\r\n            \/\/ Configure standard resilience options here\r\n        });\r\n\r\n\/\/ Use the client\r\nvar host = builder.Build();\r\nvar httpClient = host.Services\r\n    .GetRequiredService&lt;IHttpClientFactory&gt;()\r\n    .CreateClient(\"my-client\");\r\n\r\n\/\/ Make resilient HTTP request\r\nHttpResponseMessage response = await httpClient.GetAsync(\"https:\/\/jsonplaceholder.typicode.com\/comments\");<\/code><\/pre>\n<blockquote><p>To use the registered <code>my-client<\/code>, resolve <code>IHttpClientFactory<\/code> from dependency injection and use the <code>CreateClient<\/code> method to create an HTTP client.<\/p><\/blockquote>\n<p>For advanced scenarios, the APIs support building your own custom HTTP resilience pipeline:<\/p>\n<pre><code class=\"language-csharp\">services.AddHttpClient(\"my-client\")\r\n        .AddResilienceHandler(\"my-pipeline\", builder =&gt;\r\n        {\r\n            \/\/ Refer to https:\/\/www.pollydocs.org\/strategies\/retry.html#defaults for retry defaults\r\n            builder.AddRetry(new HttpRetryStrategyOptions\r\n            {\r\n                MaxRetryAttempts = 4,\r\n                Delay = TimeSpan.FromSeconds(2),\r\n                BackoffType = DelayBackoffType.Exponential\r\n            });\r\n\r\n            \/\/ Refer to https:\/\/www.pollydocs.org\/strategies\/timeout.html#defaults for timeout defaults\r\n            builder.AddTimeout(TimeSpan.FromSeconds(5));\r\n        });<\/code><\/pre>\n<h2>Why resilience is important<\/h2>\n<p>Services often rely on the HTTP protocol to make remote requests. These requests can occasionally fail due to network issues or problems on the server side. If not addressed properly, these failures can impact the service&#8217;s availability. As your service integrates more remote dependencies, the likelihood of cascading failures, where failures in one system trigger failures in others, increases. Learn more about <a href=\"https:\/\/en.wikipedia.org\/wiki\/Cascading_failure\">cascading failures<\/a>.<\/p>\n<p>To bolster the resilience of your service, especially concerning outgoing HTTP requests, you can employ several strategies:<\/p>\n<ul>\n<li><strong>Set timeouts for outgoing requests.<\/strong> After a certain period, it&#8217;s more efficient to cancel the request than to continue waiting.<\/li>\n<li><strong>Retry transient failures.<\/strong> Some requests might fail due to temporary network glitches or fleeting server-side errors. Instead of letting the operation fail, consider retrying the request.<\/li>\n<li><strong>Pause communication during remote service outages.<\/strong> If a remote service is temporarily unavailable, it may be wise to halt all communication temporarily and resume once the service is back online.<\/li>\n<li><strong>Establish fallback actions.<\/strong> Design actions to execute when primary operations fail.<\/li>\n<\/ul>\n<p>You can use any of these resilience strategies individually or combine them for outgoing HTTP requests to achieve the best results.<\/p>\n<h2>.NET resilience and Polly<\/h2>\n<p>When discussing resilience in the .NET ecosystem, one cannot overlook the <a href=\"https:\/\/github.com\/App-vNext\/Polly\">Polly library<\/a>. For years, it has been the go-to resilience solution for the .NET community. Microsoft teamed up with the Polly community to develop the new Polly v8 version, which now forms the backbone of our latest resilience libraries. The <code>Microsoft.Extensions.Http.Resilience<\/code> package provides a concise, HTTP-focused layer atop the Polly library, embracing many of Polly&#8217;s newest features. For a comprehensive look at the Polly library, please visit <a href=\"https:\/\/www.pollydocs.org\/\">pollydocs.org<\/a>.<\/p>\n<p>To use the resilience strategies mentioned earlier, there&#8217;s no additional steps required. The Polly library readily offers a variety of <a href=\"https:\/\/www.pollydocs.org\/strategies\">resilience strategies<\/a> and even lets you combine them into <a href=\"https:\/\/www.pollydocs.org\/pipelines\">resilience pipelines<\/a>.<\/p>\n<p>Here&#8217;s a simple example illustrating how to define a resilience pipeline that integrates both timeouts and retries:<\/p>\n<pre><code class=\"language-csharp\">ResiliencePipeline&lt;HttpResponseMessage&gt; pipeline = new ResiliencePipelineBuilder&lt;HttpResponseMessage&gt;()\r\n    \/\/ For retry defaults, see: https:\/\/www.pollydocs.org\/strategies\/retry.html#defaults \r\n    .AddRetry(new RetryStrategyOptions&lt;HttpResponseMessage&gt;\r\n    {\r\n        MaxRetryAttempts = 4,\r\n        Delay = TimeSpan.FromSeconds(2),\r\n        BackoffType = DelayBackoffType.Exponential\r\n    })\r\n    \/\/ For timeout defaults, see: https:\/\/www.pollydocs.org\/strategies\/timeout.html#defaults \r\n    .AddTimeout(TimeSpan.FromSeconds(5))\r\n    .Build();\r\n\r\n\/\/ Using the pipeline\r\nawait pipeline.ExecuteAsync(\r\n    async cancellationToken =&gt; await httpClient.GetAsync(\"https:\/\/jsonplaceholder.typicode.com\"), cancellationToken);<\/code><\/pre>\n<p>In the above example, we use the <code>ResiliencePipelineBuilder&lt;HttpResponseMessage&gt;<\/code> to create a pipeline. This pipeline can then support the execution of any user-defined callbacks yielding <code>HttpResponseMessage<\/code> results.<\/p>\n<h2>Resilience journey<\/h2>\n<p>Microsoft operates some of the world&#8217;s largest services. Polly is widely admired and used across many of these services. In our endeavor to unify services, we developed internal libraries that incorporate Polly, enhancing it with features like telemetry, dependency injection support, and options-based configuration. Recognizing that the broader .NET community could benefit from these enhancements, we initiated a dialogue with Polly&#8217;s maintainers. After presenting a demo integrating our improvements directly into Polly, it became evident that while these changes would enhance the Polly codebase, they would require a completely new API. This evolution culminated in the release of Polly v8, now <a href=\"https:\/\/www.thepollyproject.org\/2023\/09\/28\/polly-v8-officially-released\/\">officially available<\/a>.<\/p>\n<p>With Polly v8, a new <a href=\"https:\/\/www.nuget.org\/packages\/Polly.Core\">Polly.Core<\/a> package was launched, preserving the legacy API within the <a href=\"https:\/\/www.nuget.org\/packages\/Polly\">Polly<\/a> package. The <code>Polly.Core<\/code> package stands independently, offering minimal dependencies and retaining all the previous version&#8217;s features while introducing advancements like built-in telemetry, a zero-allocation API, unified execution, and a fluent syntax. Furthermore, the <a href=\"https:\/\/www.nuget.org\/packages\/Polly.Extensions\">Polly.Extensions<\/a> package now provides seamless integration with <a href=\"https:\/\/learn.microsoft.com\/dotnet\/core\/extensions\/dependency-injection\"><code>IServiceCollection<\/code><\/a> and facilitates the export of Polly&#8217;s telemetry events into <a href=\"https:\/\/www.pollydocs.org\/advanced\/telemetry.html#metrics\">.NET metrics<\/a>.<\/p>\n<p>The new HTTP resilience packages are build upon the foundations of Polly, presenting the .NET community with dedicated and refined HTTP-based resilience APIs.<\/p>\n<h3>Resilience packages<\/h3>\n<ul>\n<li><a href=\"https:\/\/www.nuget.org\/packages\/Microsoft.Extensions.Resilience\"><code>Microsoft.Extensions.Resilience<\/code><\/a>: This package provides a minimal set of APIs. Its primary purpose is to enrich <a href=\"https:\/\/www.pollydocs.org\/advanced\/telemetry.html#metrics\">Polly&#8217;s metrics<\/a> using the <code>AddResilienceEnricher<\/code> extension for <code>IServiceCollection<\/code>. For more details, refer to the <a href=\"https:\/\/learn.microsoft.com\/dotnet\/core\/resilience\">Resilience docs<\/a>.<\/li>\n<li><a href=\"https:\/\/www.nuget.org\/packages\/Microsoft.Extensions.Http.Resilience\"><code>Microsoft.Extensions.Http.Resilience<\/code><\/a>: This package offers HTTP-specific APIs that integrate with Polly v8 and <code>IHttpClientFactory<\/code>. It is the successor to the <code>Microsoft.Extensions.Http.Polly<\/code> package and is recommended for all new projects. See the section below for more information.<\/li>\n<li><a href=\"https:\/\/www.nuget.org\/packages\/Microsoft.Extensions.Http.Polly\"><code>Microsoft.Extensions.Http.Polly<\/code><\/a>: This package integrates older versions of Polly with <a href=\"https:\/\/learn.microsoft.com\/dotnet\/fundamentals\/networking\/http\/httpclient\"><code>HttpClient<\/code><\/a> and <a href=\"https:\/\/learn.microsoft.com\/dotnet\/core\/extensions\/httpclient-factory\"><code>IHttpClientFactory<\/code><\/a>.<\/li>\n<\/ul>\n<h2>Resilient HTTP requests<\/h2>\n<p>To execute resilient HTTP requests, first install the <a href=\"https:\/\/www.nuget.org\/packages\/Microsoft.Extensions.Http.Resilience\"><code>Microsoft.Extensions.Http.Resilience<\/code><\/a> package. This package exposes the following extensions for <a href=\"https:\/\/learn.microsoft.com\/dotnet\/core\/extensions\/httpclient-factory\"><code>IHttpClientBuilder<\/code><\/a>:<\/p>\n<ul>\n<li><strong><code>AddStandardResilienceHandler<\/code><\/strong>: This adds a resilience handler with a standard resilience pipeline suitable for most scenarios.<\/li>\n<li><strong><code>AddStandardHedgingHandler<\/code><\/strong>: This introduces a resilience handler with a standard hedging pipeline, which supports requests to multiple endpoints.<\/li>\n<li><strong><code>AddResilienceHandler<\/code><\/strong>: This incorporates a resilience handler that allows configuration of resilience strategies in the resilience pipeline.<\/li>\n<\/ul>\n<p>For further information, visit the official <a href=\"https:\/\/learn.microsoft.com\/dotnet\/core\/resilience\/http-resilience\">Build Resilient HTTP Apps<\/a> documentation.<\/p>\n<h2>Standard resilience pipeline<\/h2>\n<p>The standard pipeline is the recommended resilience API to use. It combines five Polly strategies to form a resilience pipeline suitable for most scenarios. The standard pipeline contains the following strategies executed in the order below:<\/p>\n<table>\n<thead>\n<tr>\n<th style=\"text-align: right;\">Order<\/th>\n<th>Strategy<\/th>\n<th>Description<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td style=\"text-align: right;\"><strong>1<\/strong><\/td>\n<td>Rate limiter<\/td>\n<td>The rate limiter pipeline limits the maximum number of concurrent requests being sent to the dependency.<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: right;\"><strong>2<\/strong><\/td>\n<td>Total request timeout<\/td>\n<td>The total request timeout pipeline applies an overall timeout to the execution, ensuring that the request, including retry attempts, doesn&#8217;t exceed the configured limit.<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: right;\"><strong>3<\/strong><\/td>\n<td>Retry<\/td>\n<td>The retry pipeline retries the request in case the dependency is slow or returns a transient error.<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: right;\"><strong>4<\/strong><\/td>\n<td>Circuit breaker<\/td>\n<td>The circuit breaker blocks the execution if too many direct failures or timeouts are detected.<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: right;\"><strong>5<\/strong><\/td>\n<td>Attempt timeout<\/td>\n<td>The attempt timeout pipeline limits each request attempt duration and throws if it&#8217;s exceeded.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>To integrate the standard pipeline, use the <code>AddStandardResilienceHandler<\/code> extension for <code>IHttpClientBuilder<\/code>:<\/p>\n<pre><code class=\"language-csharp\">\/\/ Add standard resilience handler to the HTTP client\r\nservices.AddHttpClient(\"my-client\")\r\n        .AddStandardResilienceHandler();<\/code><\/pre>\n<blockquote><p>Invoking <code>AddStandardResilienceHandler<\/code> returns an <code>IHttpStandardResiliencePipelineBuilder<\/code> instance which exposes additional extensions for configuring the standard pipeline.<\/p><\/blockquote>\n<p>The previous example uses default options where for both retries and circuit breaker strategies, the following outcomes are handled:<\/p>\n<ul>\n<li>Any status code <em>500<\/em> or above.<\/li>\n<li><em>429<\/em> (Too Many Requests).<\/li>\n<li><em>408<\/em> (Request Timeout).<\/li>\n<li>Exceptions: <code>HttpRequestException<\/code> and <code>TimeoutRejectedException<\/code>.<\/li>\n<\/ul>\n<p>Customizing the standard pipeline options when calling <code>AddStandardResilienceHandler<\/code> is also possible:<\/p>\n<pre><code class=\"language-csharp\">services\r\n    .AddHttpClient(\"my-client\")\r\n    .AddStandardResilienceHandler(options =&gt;\r\n    {\r\n        \/\/ Customize retry\r\n        options.Retry.ShouldHandle = new PredicateBuilder&lt;HttpResponseMessage&gt;()\r\n            .Handle&lt;TimeoutRejectedException&gt;()\r\n            .Handle&lt;HttpRequestException&gt;()\r\n            .HandleResult(response =&gt; response.StatusCode == HttpStatusCode.InternalServerError);\r\n        options.Retry.MaxRetryAttempts = 5;\r\n\r\n        \/\/ Customize attempt timeout\r\n        options.AttemptTimeout.Timeout = TimeSpan.FromSeconds(2);\r\n    });<\/code><\/pre>\n<p>Alternatively, configure the options with the <code>Configure<\/code> extensions:<\/p>\n<pre><code class=\"language-csharp\">\/\/ Retrieve the IConfiguration\r\nIConfiguration configuration = ...; \r\n\r\nservices\r\n    .AddHttpClient(\"my-client\")\r\n    .AddStandardResilienceHandler()\r\n    \/\/ Configure the standard options with IConfiguration\r\n    .Configure(configuration.GetSection(\"my-section\"))\r\n    \/\/ Or using callbacks\r\n    .Configure(options =&gt; \r\n    {\r\n        \/\/ Additional configuration\r\n    });<\/code><\/pre>\n<p>The previous example:<\/p>\n<ul>\n<li>Employs <code>Configure<\/code> that takes <code>IConfiguration<\/code> to bind options from <em>my-section<\/em>.<\/li>\n<li>Uses another <code>Configure<\/code> showcasing the support for configuration chaining.<\/li>\n<\/ul>\n<blockquote><p>The standard resilience handler supports dynamic reloading of options. If the <code>configuration<\/code> changes, the resilience pipeline dynamically refreshes, using the new configuration for request handling. This enhancement is enabled by the <a href=\"https:\/\/www.pollydocs.org\/advanced\/dependency-injection.html#dynamic-reloads\">dynamic reloads<\/a> feature of the Polly library.<\/p><\/blockquote>\n<h2>Standard hedging pipeline<\/h2>\n<p>This is a new addition to the resilience family. The standard hedging handler is similar to the standard resilience handler. However, instead of using a retry strategy, this pipeline employs a hedging strategy. Introduced in Polly v8, the hedging strategy aims to improve request latency by issuing multiple concurrent requests. See Polly&#8217;s official <a href=\"https:\/\/www.pollydocs.org\/strategies\/hedging.html\">hedging strategy documentation<\/a> for more details.<\/p>\n<p>The standard hedging pipeline consists of the following strategies:<\/p>\n<table>\n<thead>\n<tr>\n<th style=\"text-align: right;\">Order<\/th>\n<th>Strategy<\/th>\n<th>Description<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td style=\"text-align: right;\"><strong>1<\/strong><\/td>\n<td>Total request timeout<\/td>\n<td>The total request timeout pipeline applies an overall timeout to the execution, ensuring that the request, including hedging attempts, doesn&#8217;t exceed the configured limit.<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: right;\"><strong>2<\/strong><\/td>\n<td>Hedging<\/td>\n<td>The hedging strategy executes the requests against multiple endpoints in case the dependency is slow or returns a transient error. Routing is options, by default it just hedges the URL provided by the original .<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: right;\"><strong>3<\/strong><\/td>\n<td>Rate limiter (per endpoint)<\/td>\n<td>The rate limiter pipeline limits the maximum number of concurrent requests being sent to the dependency.<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: right;\"><strong>4<\/strong><\/td>\n<td>Circuit breaker (per endpoint)<\/td>\n<td>The circuit breaker blocks the execution if too many direct failures or timeouts are detected.<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: right;\"><strong>5<\/strong><\/td>\n<td>Attempt timeout (per endpoint)<\/td>\n<td>The attempt timeout pipeline limits each request attempt duration and throws if it&#8217;s exceeded.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>To use standard hedging:<\/p>\n<pre><code class=\"language-csharp\">services.AddHttpClient(\"my-client\")\r\n        .AddStandardHedgingHandler();<\/code><\/pre>\n<p>By default, hedging sends another request if no response is received within 2 seconds. It then waits for the quickest request to complete.<\/p>\n<p>Standard hedging uses a pool of circuit breakers to ensure that requests aren&#8217;t sent to unhealthy endpoints. Typically, the pool selection is based on the URL authority (scheme + host + port). In the provided example, no routing is defined for hedging, so all requests go to the URL specified in the request message.<\/p>\n<p>To customize the selection of circuit breakers, use the <code>SelectPipelineBy<\/code> extension method:<\/p>\n<pre><code class=\"language-csharp\">services.AddHttpClient(\"my-client\")\r\n        .AddStandardHedgingHandler()\r\n        .SelectPipelineBy(serviceProvider =&gt; request =&gt; request.RequestUri.Host);<\/code><\/pre>\n<p><code>SelectPipelineBy<\/code> requires a factory that, when invoked, returns a function to retrieve a string from <code>HttpRequestMessage<\/code>. This string is then used to pool the circuit breakers.<\/p>\n<p>Like the standard pipeline, you can also configure standard hedging options:<\/p>\n<pre><code class=\"language-csharp\">services\r\n    .AddHttpClient(\"my-client\")\r\n    .AddStandardHedgingHandler()\r\n    .Configure(configuration.GetSection(\"my-section\"))\r\n    .Configure(options =&gt; \r\n    {\r\n        options.Hedging.MaxHedgedAttempts = 3;\r\n        options.Hedging.Delay = TimeSpan.FromSeconds(1);\r\n    });<\/code><\/pre>\n<p>In the example above:<\/p>\n<ul>\n<li><code>Configure<\/code> binds options from <em>my-section<\/em> using <code>IConfiguration<\/code>.<\/li>\n<li>Another <code>Configure<\/code> demonstrates configuration chaining and adjusts the maximum number of hedged attempts to 3. This means the hedging strategy can make up to 4 requests to a remote endpoint with a 1-second delay between them.<\/li>\n<\/ul>\n<h3>Standard hedging and routing<\/h3>\n<p>One powerful feature of the standard hedging pipeline is its ability to configure URL routes for requests. This enables hedging to send requests to different endpoints in case some are unresponsive or unhealthy, as demonstrated below:<\/p>\n<pre><code class=\"language-csharp\">services\r\n    .AddHttpClient(\"my-client\")\r\n    .AddStandardHedgingHandler(routingBuilder =&gt;\r\n    {\r\n        routingBuilder.ConfigureOrderedGroups(options =&gt;\r\n        {\r\n            options.Groups.Add(new UriEndpointGroup\r\n            {\r\n                Endpoints =\r\n                {\r\n                    new() { Uri = new(\"https:\/\/example.net\/api\/a\"), Weight = 95 },\r\n                    new() { Uri = new(\"https:\/\/example.net\/api\/b\"), Weight = 5 },\r\n                }\r\n            });\r\n\r\n            options.Groups.Add(new UriEndpointGroup\r\n            {\r\n                Endpoints =\r\n                {\r\n                    new() { Uri = new(\"https:\/\/example.net\/api\/c\"), Weight = 95 },\r\n                    new() { Uri = new(\"https:\/\/example.net\/api\/d\"), Weight = 5 },\r\n                }\r\n            });            \r\n        });\r\n    });<\/code><\/pre>\n<p>In this example:<\/p>\n<ul>\n<li><code>AddStandardHedgingHandler<\/code> introduces standard hedging.<\/li>\n<li>Routes for hedging are set up within <code>AddStandardHedgingHandler<\/code> using the <code>ConfigureOrderedGroups<\/code> extension.<\/li>\n<li>Two groups with multiple endpoints are added. With ordered groups, every request selects a single endpoint from each group sequentially. After exhausting all groups, hedging stops, even if <code>MaxHedgedAttempts<\/code> is not met.<\/li>\n<li>The <code>Weight<\/code> property indicates the probability of selecting that endpoint. In the example above, there is a <em>95%<\/em> chance of selecting the endpoint <code>https:\/\/example.net\/api\/a<\/code> and a <em>5%<\/em> chance for the <code>https:\/\/example.net\/api\/b<\/code> endpoint.<\/li>\n<\/ul>\n<blockquote><p>While the example employs ordered groups via <code>ConfigureOrderedGroups<\/code>, the API also offers <code>ConfigureWeightedGroups<\/code>, which permits group selection based on weight.<\/p><\/blockquote>\n<p>Visit the official <a href=\"https:\/\/learn.microsoft.com\/dotnet\/core\/resilience\/http-resilience?tabs=dotnet-cli#customize-hedging-handler-route-selection\">customize hedging handler route selection<\/a> documentation to learn more about routing in hedging.<\/p>\n<h3>Standard hedging and unavailable endpoints<\/h3>\n<p>Consider this configuration:<\/p>\n<pre><code class=\"language-csharp\">services\r\n    .AddHttpClient(\"my-client\", client =&gt;\r\n    {\r\n        client.Timeout = TimeSpan.FromSeconds(10);\r\n        client.BaseAddress = new Uri(\"https:\/\/example.net\");\r\n    })\r\n    .AddStandardHedgingHandler(routingBuilder =&gt;\r\n    {\r\n        routingBuilder.ConfigureOrderedGroups(options =&gt;\r\n        {\r\n            options.Groups.Add(new UriEndpointGroup\r\n            {\r\n                Endpoints = [new() { Uri = new(\"https:\/\/jsonplaceholder.typicode.com:999\") }] \/\/ Unavailable endpoint\r\n            });\r\n\r\n            options.Groups.Add(new UriEndpointGroup\r\n            {\r\n                Endpoints = [new() { Uri = new(\"https:\/\/jsonplaceholder.typicode.com\") }]\r\n            });\r\n        });\r\n    })\r\n    .Configure(options =&gt; \r\n    {\r\n        options.Endpoint.CircuitBreaker.MinimumThroughput = 5;\r\n        options.Endpoint.CircuitBreaker.SamplingDuration = TimeSpan.FromSeconds(5);\r\n        options.Endpoint.Timeout.Timeout = TimeSpan.FromSeconds(1);\r\n    });<\/code><\/pre>\n<p>Note the inaccessible URL <code>https:\/\/jsonplaceholder.typicode.com:999<\/code> representing an unhealthy endpoint.<\/p>\n<p>Using the above-configured HTTP client:<\/p>\n<pre><code class=\"language-csharp\">\/\/ Create the client\r\nHttpClient client = services.BuildServiceProvider()\r\n    .GetRequiredService&lt;IHttpClientFactory&gt;()\r\n    .CreateClient(\"my-client\");\r\n\r\n\/\/ Use the client\r\nfor (int i = 0; i &lt; 10; i++)\r\n{\r\n    var watch = Stopwatch.StartNew();\r\n    await client.GetStringAsync(\"posts\");\r\n    Console.WriteLine(\"{0}: {1}ms\", i + 1, watch.ElapsedMilliseconds);\r\n}<\/code><\/pre>\n<p>Results in the following console output:<\/p>\n<pre><code class=\"language-bash\">1: 1396ms\r\n2: 1024ms\r\n3: 1015ms\r\n4: 1028ms\r\n5: 1015ms\r\n6: 14ms\r\n7: 11ms\r\n8: 15ms\r\n9: 15ms\r\n10: 13ms<\/code><\/pre>\n<p>Initial attempts are lengthy since hedging tries to obtain a response from the unavailable endpoint. After a duration aligning with the circuit breaker&#8217;s <code>SamplingDuration<\/code>, the pipeline identifies the malfunctioning endpoint and opens its circuit breaker. When the circuit breaker is open, attempts to reach the first endpoint instantly fail, prompting an immediate request to a secondary endpoint, hence the faster subsequent attempts.<\/p>\n<p>See <a href=\"https:\/\/www.pollydocs.org\/strategies\/circuit-breaker.html#state-diagram\">circuit breaker state diagram<\/a> to learn more about states of circuit breaker and the state transitions.<\/p>\n<blockquote><p>In situations like the one described above, after a certain duration, you might observe increased latency for a single request. In terms of circuit breakers, this is a probing request checking endpoint health. Probing requests might exhibit increased latencies. If probing is successful, the circuit breaker is closed, allowing hedging to reconnect with the primary endpoint.<\/p><\/blockquote>\n<h2>Custom resilience pipeline<\/h2>\n<p>There are scenarios where the standard resilience or standard hedging pipeline might not be suitable. In such cases, you have APIs that allow you to build your own HTTP-based resilience pipeline. These APIs integrate seamlessly with the Polly library and support all built-in resilience strategies.<\/p>\n<p>To add a custom resilience pipeline, use the <code>AddResilienceHandler<\/code> extension for <code>IHttpClientBuilder<\/code>:<\/p>\n<pre><code class=\"language-csharp\">services\r\n    .AddHttpClient(\"my-client\")\r\n    .AddResilienceHandler(\"custom-pipeline\", builder =&gt;\r\n    {\r\n        builder\r\n            .AddRetry(new HttpRetryStrategyOptions())\r\n            .AddTimeout(new HttpTimeoutStrategyOptions());\r\n    });<\/code><\/pre>\n<p>In the example above:<\/p>\n<ul>\n<li>A new named <em>my-client<\/em> HTTP Client is registered.<\/li>\n<li><code>AddResilienceHandler<\/code> is called to add a resilience handler that uses a resilience pipeline, configured by calling extensions on <code>builder<\/code>.<\/li>\n<li>Both retry and timeout strategies are added to the pipeline.<\/li>\n<\/ul>\n<h3>Custom resilience pipeline and dynamic reloads<\/h3>\n<p>The custom resilience pipeline supports dynamic reloads, i.e., it transparently refreshes the pipeline whenever the options are changed. To enable dynamic reloads, the example above can be rewritten as:<\/p>\n<pre><code class=\"language-csharp\">\/\/ 1. Define options that represent the custom pipeline\r\npublic class CustomPipelineOptions\r\n{\r\n    [Required]\r\n    public HttpRetryStrategyOptions Retry { get; set; } = new();\r\n\r\n    [Required]\r\n    public HttpTimeoutStrategyOptions Timeout { get; set; } = new();\r\n}\r\n\r\n\/\/ 2. Build a configuration that dynamically reloads\r\nvar configuration = new ConfigurationBuilder()\r\n    .AddJsonFile(\"appsettings.json\", optional: false, reloadOnChange: true)\r\n    .Build();\r\n\r\n\/\/ 3. Bind named CustomPipelineOptions from configuration\r\nvar services = new ServiceCollection();\r\nservices.AddLogging(b =&gt; b.AddConsole().SetMinimumLevel(LogLevel.Debug));\r\nservices.Configure&lt;CustomPipelineOptions&gt;(\"custom-pipeline\", configuration.GetRequiredSection(\"CustomPipeline\"));\r\n\r\n\/\/ 4. Define the HTTP pipeline\r\nservices\r\n    .AddHttpClient(\"my-client\", client =&gt; client.BaseAddress = new Uri(\"https:\/\/jsonplaceholder.typicode.com\"))\r\n    .AddResilienceHandler(\"custom-pipeline\", (builder, context) =&gt;\r\n    {\r\n        \/\/ Enable dynamic reloads of this pipeline whenever the named CustomPipelineOptions change\r\n        context.EnableReloads&lt;CustomPipelineOptions&gt;(\"custom-pipeline\");\r\n\r\n        \/\/ Retrieve the named options\r\n        var options = context.GetOptions&lt;CustomPipelineOptions&gt;(\"custom-pipeline\");\r\n\r\n        builder\r\n            .AddRetry(options.Retry)\r\n            .AddTimeout(options.Timeout);\r\n    });<\/code><\/pre>\n<p>The <code>appsettings.json<\/code> file looks similar to:<\/p>\n<pre><code class=\"language-json\">{\r\n    \"CustomPipeline\": {\r\n        \"Retry\": {\r\n            \"ShouldRetryAfterHeader\": false,\r\n            \"MaxRetryAttempts\": 3,\r\n            \"BackoffType\": 2,\r\n            \"UseJitter\": true,\r\n            \"Delay\": \"00:00:02\",\r\n          },\r\n          \"Timeout\": {\r\n            \"Timeout\": \"00:00:30\",\r\n          }\r\n    }\r\n}<\/code><\/pre>\n<p>To test the reloads, run the application and use the defined HTTP pipeline:<\/p>\n<pre><code class=\"language-csharp\">HttpClient client = services.BuildServiceProvider()\r\n    .GetRequiredService&lt;IHttpClientFactory&gt;()\r\n    .CreateClient(\"my-client\");\r\n\r\nawait client.GetStringAsync(\"posts\");\r\nConsole.ReadLine();<\/code><\/pre>\n<p>Now, whenever you modify the <code>appsettings.json<\/code>, the pipeline is dynamically reloaded. You should see the following Polly event in the console:<\/p>\n<pre><code class=\"language-bash\">info: Polly[0]\r\n      Resilience event occurred. EventName: 'OnReload', Source: 'my-client-custom-pipeline\/\/(null)', Operation Key: '', Result: ''<\/code><\/pre>\n<blockquote><p>Dynamic reloads are automatically enabled for both the standard resilience pipeline and the standard hedging pipeline.<\/p><\/blockquote>\n<p>Visit the official <a href=\"https:\/\/learn.microsoft.com\/dotnet\/core\/resilience\/http-resilience#dynamic-reload\">dynamic reload<\/a> documentation to learn more.<\/p>\n<h3>HTTP resilience options<\/h3>\n<p>The <code>Microsoft.Extensions.Http.Resilience<\/code> library defines HTTP-specific options tailored for HTTP scenarios. These options extend those defined in the Polly library and modify some of the defaults. Specifically, the <code>ShouldHandle<\/code> predicate of reactive strategies has been updated to retry on specific HTTP errors.<\/p>\n<p>Below is a table showing the HTTP-specific options:<\/p>\n<table>\n<thead>\n<tr>\n<th style=\"text-align: right;\">Options<\/th>\n<th>Base Options<\/th>\n<th>Notes<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td style=\"text-align: right;\"><code>HttpRetryStrategyOptions<\/code><\/td>\n<td><a href=\"https:\/\/www.pollydocs.org\/strategies\/retry#defaults\"><code>RetryStrategyOptions&lt;HttpResponseMessage&gt;<\/code><\/a><\/td>\n<td>Some defaults have been changed. <sup>1<\/sup><\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: right;\"><code>HttpCircuitBreakerStrategyOptions<\/code><\/td>\n<td><a href=\"https:\/\/www.pollydocs.org\/strategies\/circuit-breaker#defaults\"><code>CircuitBreakerStrategyOptions&lt;HttpResponseMessage&gt;<\/code><\/a><\/td>\n<td>Some defaults have been changed. <sup>2<\/sup><\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: right;\"><code>HttpHedgingStrategyOptions<\/code><\/td>\n<td><a href=\"https:\/\/www.pollydocs.org\/strategies\/hedging#defaults\"><code>HedgingStrategyOptions&lt;HttpResponseMessage&gt;<\/code><\/a><\/td>\n<td>Some defaults have been changed. <sup>3<\/sup><\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: right;\"><code>HttpRateLimiterStrategyOptions<\/code><\/td>\n<td><a href=\"https:\/\/www.pollydocs.org\/strategies\/rate-limiter#defaults\"><code>RateLimiterStrategyOptions<\/code><\/a><\/td>\n<td>Defaults remain unchanged.<\/td>\n<\/tr>\n<tr>\n<td style=\"text-align: right;\"><code>HttpTimeoutStrategyOptions<\/code><\/td>\n<td><a href=\"https:\/\/www.pollydocs.org\/strategies\/timeout#defaults\"><code>TimeoutStrategyOptions<\/code><\/a><\/td>\n<td>Defaults remain unchanged.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><sup>1<\/sup> The <code>HttpRetryStrategyOptions<\/code> uses an exponential backoff type with jitter and handles the <em>Retry-After<\/em> header automatically. The <code>ShouldHandle<\/code> predicate addresses any status code of <em>500<\/em> or above, as well as <em>429<\/em> and <em>408<\/em> status codes. Additionally, both <code>HttpRequestException<\/code> and <code>TimeoutRejectedException<\/code> are retried.<\/p>\n<p><sup>2<\/sup> The circuit breaker&#8217;s <code>ShouldHandle<\/code> predicate addresses any status code of <em>500<\/em> or above, as well as <em>429<\/em> and <em>408<\/em> status codes. Moreover, <code>HttpRequestException<\/code> and <code>TimeoutRejectedException<\/code> are handled.<\/p>\n<p><sup>3<\/sup> The <code>ShouldHandle<\/code> predicate for hedging addresses any status code of <em>500<\/em> or above, as well as <em>429<\/em> and <em>408<\/em> status codes. Additionally, <code>HttpRequestException<\/code> and <code>TimeoutRejectedException<\/code> are hedged.<\/p>\n<h2>Performance<\/h2>\n<p>The new resilience APIs are built on Polly v8, which was designed from the ground up to support zero-allocations and enhanced performance. To illustrate these improvements, let&#8217;s compare the performance of the standard resilience pipeline in Polly v7 to that in Polly v8:<\/p>\n<pre><code class=\"language-markdown\">|                    Method |     Mean |     Error |    StdDev | Ratio |   Gen0 | Allocated | Alloc Ratio |\r\n|-------------------------- |---------:|----------:|----------:|------:|-------:|----------:|------------:|\r\n| StandardPipeline_Polly_V7 | 3.236 us | 0.0130 us | 0.0187 us |  1.00 | 0.1488 |    3816 B |        1.00 |\r\n| StandardPipeline_Polly_V8 | 3.104 us | 0.0237 us | 0.0317 us |  0.96 | 0.0381 |    1008 B |        0.26 |<\/code><\/pre>\n<p>While the execution time is marginally faster, the APIs built on Polly v8 use almost 4x less memory.<\/p>\n<h2>Summary<\/h2>\n<p>The .NET 8 introduces new <code>Microsoft.Extensions.Http.Resilience<\/code> and <code>Microsoft.Extensions.Resilience<\/code> packages, built on the Polly library. These new packages allow developers to integrate resilience strategies seamlessly into the HTTP pipeline. Collaboration with the Polly community led to the development of Polly v8, which offers improved performance, built-in telemetry, and a fluent syntax. These advancements simplify the integration of resilience for developers while ensuring efficiency and reduced memory usage.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>How to integrate resilience and into HTTP Client<\/p>\n","protected":false},"author":133391,"featured_media":49328,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[685,7699,7509,756,7591],"tags":[4,46,7539,7676,80,7770,7772,7769,7771],"class_list":["post-49135","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet","category-dotnet-fundamentals","category-aspnetcore","category-csharp","category-networking","tag-net","tag-c","tag-cloud","tag-http","tag-httpclient","tag-ihttpclientfactory","tag-polly","tag-resilience","tag-scale"],"acf":[],"blog_post_summary":"<p>How to integrate resilience and into HTTP Client<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/49135","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/133391"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=49135"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/49135\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/49328"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=49135"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=49135"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=49135"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}