Re-reading ASP.Net Core request bodies with EnableBuffering()

Avatar

Jeremy

In some scenarios there’s a need to read the request body multiple times. Some examples include

  • Logging the raw requests to replay in load test environment
  • Middleware that read the request body multiple times to process it

Usually Request.Body does not support rewinding, so it can only be read once. A straightforward solution is to save a copy of the stream in another stream that supports seeking so the content can be read multiple times from the copy.

In ASP.NET framework it was possible to read the body of an HTTP request multiple times using HttpRequest.GetBufferedInputStream method. However, in ASP.NET Core a different approach must be used.

In ASP.NET Core 2.1 we added an extension method EnableBuffering() for HttpRequest. This is the suggested way to enable request body for multiple reads. Here is an example usage in the InvokeAsync() method of a custom ASP.NET middleware:

public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
    context.Request.EnableBuffering();

    // Leave the body open so the next middleware can read it.
    using (var reader = new StreamReader(
        context.Request.Body,
        encoding: Encoding.UTF8,
        detectEncodingFromByteOrderMarks: false,
        bufferSize: bufferSize,
        leaveOpen: true))
    {
        var body = await reader.ReadToEndAsync();
        // Do some processing with body…

        // Reset the request body stream position so the next middleware can read it
        context.Request.Body.Position = 0;
    }

    // Call the next delegate/middleware in the pipeline
    await next(context);
}

The backing FileBufferingReadStream uses memory stream of a certain size first then falls back to a temporary file stream. By default the size of the memory stream is 30KB. There are also other EnableBuffering() overloads that allow specifying a different threshold, and/or a limit for the total size:

public static void EnableBuffering(this HttpRequest request, int bufferThreshold)

public static void EnableBuffering(this HttpRequest request, long bufferLimit)

public static void EnableBuffering(this HttpRequest request, int bufferThreshold, long bufferLimit)

For example, a call of

context.Request.EnableBuffering(bufferThreshold: 1024 * 45, bufferLimit: 1024 * 100);

enables a read buffer with limit of 100KB. Data is buffered in memory until the content exceeds 45KB, then it’s moved to a temporary file. By default there’s no limit on the buffer size but if there’s one specified and the content of request body exceeds the limit, an System.IOException will be thrown.

These overloads offer flexibility if there’s a need to fine-tune the buffering behaviors. Just keep in mind that:

  • Even though the memory stream is rented from a pool, it still has memory cost associated with it.
  • After the read is over the bufferThreshold the performance will be slower since a file stream will be used.
Avatar
Jeremy Meng

Software Development Engineer

Follow Jeremy   

12 comments

  • Avatar
    Grieger,Jacob

    Where does the signature "public async Task InvokeAsync(HttpContext context, RequestDelegate next)" come from? In the Microsoft docs the custom middleware implementation examples do have a constructor taking the RequestDelegate. Without this constructor I get runtime errors.

  • Avatar
    Satyadeep Behera

    Hi Jeremy,In one of my project I need to implement reading the header data each time multiple numbers, tried to do using the way you told.Problem happense during DI.I’ve done following way to call:
    from startup.csapp.Use(typeof(IdentityValidation));using AppFunc = Func<IDictionary<string, object>, Task>;public class IdentityValidation{
    private readonly AppFunc _next;public IdentityValidation(AppFunc next){_next = next;}
    }Problem here, I’m unable to inject depedency along with delegate call, it throws complie time error,can you please guide how do you call “Invoke” mthod?
    Thanks,Satyadeep

  • Avatar
    Marcos Assis

    Hi Jeremy,
    I’m a little confused, in one of my projects I’m using the extension method Request.EnableRewind() from namespace Microsoft.AspNetCore.Http.Internal. Is it the same thing?

    • Lindsey Crowe
      Lindsey Crowe

      Hey Marcos, did you ever find the answer to ths question? I’m keen to know the same, or at least a clear comparison of capabilities…

  • Avatar
    Michael Spranger

    Is there a related mechanism for the response body?
    We are logging the response, too. However, this requires a rather convoluted approach of replacing the original response stream in context.Response.Body with a MemoryStream before calling await _next(context); , while keeping a reference to the former. After logging the response, we need to copy the content from the MemoryStream into the original stream and place it back into context.Response.Body.

  • Avatar
    Jonas Granlund

    Hi,
    Great Explanation.
    Is there any possiblility to get the deserialized request object from middleware after the response leave the controller through the middleware towards client?

    In Fullframework we solved it like this and listend to the types about to be disposed which is far from optimal, but something similar might apply in Core?


    private object getDeserializedRequestObject(HttpResponseMessage res)
    {
    object reqObj = null;
    if (res?.RequestMessage?.Properties?.ContainsKey("MS_DisposableRequestResources") == true)
    {
    List lst = res.RequestMessage.Properties["MS_DisposableRequestResources"] as List;
    var cntr = lst.Where(x => x is ApiController).FirstOrDefault();
    if (cntr != null)
    {
    var aargs = (cntr as ApiController)?.ActionContext?.ActionArguments;

    if (aargs != null && aargs.Count == 1)
    {
    reqObj = aargs.First();
    if (reqObj is KeyValuePair<string, object>)
    {
    reqObj = ((KeyValuePair<string, object>)reqObj).Value;
    }
    }
    else
    reqObj = aargs;

    aargs = null;
    lst = null;
    cntr = null;
    }
    }
    return reqObj;
    }

Leave a comment