March 27th, 2019

Re-reading ASP.Net Core request bodies with EnableBuffering()

Jeremy Caney
Jeremy Caney

In some scenarios there’s a need to read the request body multiple times. Some examples include

  • Logging the raw requests to replay in load test environment
  • Middleware that read the request body multiple times to process it

Usually Request.Body does not support rewinding, so it can only be read once. A straightforward solution is to save a copy of the stream in another stream that supports seeking so the content can be read multiple times from the copy.

In ASP.NET framework it was possible to read the body of an HTTP request multiple times using HttpRequest.GetBufferedInputStream method. However, in ASP.NET Core a different approach must be used.

In ASP.NET Core 2.1 we added an extension method EnableBuffering() for HttpRequest. This is the suggested way to enable request body for multiple reads. Here is an example usage in the InvokeAsync() method of a custom ASP.NET middleware:

public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
    context.Request.EnableBuffering();

    // Leave the body open so the next middleware can read it.
    using (var reader = new StreamReader(
        context.Request.Body,
        encoding: Encoding.UTF8,
        detectEncodingFromByteOrderMarks: false,
        bufferSize: bufferSize,
        leaveOpen: true))
    {
        var body = await reader.ReadToEndAsync();
        // Do some processing with body…

        // Reset the request body stream position so the next middleware can read it
        context.Request.Body.Position = 0;
    }

    // Call the next delegate/middleware in the pipeline
    await next(context);
}

The backing FileBufferingReadStream uses memory stream of a certain size first then falls back to a temporary file stream. By default the size of the memory stream is 30KB. There are also other EnableBuffering() overloads that allow specifying a different threshold, and/or a limit for the total size:

public static void EnableBuffering(this HttpRequest request, int bufferThreshold)

public static void EnableBuffering(this HttpRequest request, long bufferLimit)

public static void EnableBuffering(this HttpRequest request, int bufferThreshold, long bufferLimit)

For example, a call of

context.Request.EnableBuffering(bufferThreshold: 1024 * 45, bufferLimit: 1024 * 100);

enables a read buffer with limit of 100KB. Data is buffered in memory until the content exceeds 45KB, then it’s moved to a temporary file. By default there’s no limit on the buffer size but if there’s one specified and the content of request body exceeds the limit, an System.IOException will be thrown.

These overloads offer flexibility if there’s a need to fine-tune the buffering behaviors. Just keep in mind that:

  • Even though the memory stream is rented from a pool, it still has memory cost associated with it.
  • After the read is over the bufferThreshold the performance will be slower since a file stream will be used.
Category
ASP.NET

Author

13 comments

Discussion is closed. Login to edit/delete existing comments.

Newest
Newest Popular Oldest
  • Jeremy Conterio

    Is it possible to get a handle to the temp file, so we don’t have to create our own local copy, to save time?

    We also have limited space on the hard disk so we would prefer to have just one copy instead of two.

  • Jonas Granlund

    Hi,
    Great Explanation.
    Is there any possiblility to get the deserialized request object from middleware after the response leave the controller through the middleware towards client?

    In Fullframework we solved it like this and listend to the types about to be disposed which is far from optimal, but something similar might apply in Core?

    private object getDeserializedRequestObject(HttpResponseMessage res)
    {
    object reqObj = null;
    if (res?.RequestMessage?.Properties?.ContainsKey("MS_DisposableRequestResources") == true)
    ...

    Read more
  • Vojta Machacek

    What should be the bufferSize?

    • Duco Winterwerp

      1024 apparently. Source

  • Michael Spranger

    Is there a related mechanism for the response body?
    We are logging the response, too. However, this requires a rather convoluted approach of replacing the original response stream in context.Response.Body with a MemoryStream before calling await _next(context); , while keeping a reference to the former. After logging the response, we need to copy the content from the MemoryStream into the original stream and place it back into context.Response.Body.

  • Marcos Assis

    Hi Jeremy,
    I’m a little confused, in one of my projects I’m using the extension method Request.EnableRewind() from namespace Microsoft.AspNetCore.Http.Internal. Is it the same thing?

  • preeti agarwal

    Great article on Re-reading ASP.Net Core request bodies with EnableBuffering()
    code you explained is good and it will helpful for me 
    keep sharing more blog
    https://www.exltech.in/dot-net-training.html

  • Satyadeep Behera

    Hi Jeremy,In one of my project I need to implement reading the header data each time multiple numbers, tried to do using the way you told.Problem happense during DI.I’ve done following way to call:
    from startup.csapp.Use(typeof(IdentityValidation));using AppFunc = Func<IDictionary<string, object>, Task>;public class IdentityValidation{
    private readonly AppFunc _next;public IdentityValidation(AppFunc next){_next = next;}
    }Problem here, I’m unable to inject depedency along with delegate call, it throws complie time error,can you please guide how do you call “Invoke” mthod?
    Thanks,Satyadeep

    • David FowlerMicrosoft employee

      I suggest asking this question on Stackoverflow

  • Grieger,Jacob

    Where does the signature "public async Task InvokeAsync(HttpContext context, RequestDelegate next)" come from? In the Microsoft docs the custom middleware implementation examples do have a constructor taking the RequestDelegate. Without this constructor I get runtime errors.

Read next