Passing OData Query Options in the Request Body

John Gathogo

December 4th, 202012 1

The query options part of an OData URL can be quite long, potentially exceeding the maximum length of URLs supported by components involved in transmitting or processing the request. HTTP does not impose any limits on the length of a URL, however, many hosting environments (including IIS) impose limitations which may restrict the client’s ability to request the exact set of data that they are interested in. These limitations would especially pose a big challenge for clients using explicit select statements for example.

Background

One way that you can go round these limitations is by wrapping the GET request in a batch request. This approach requires you to be conversant with how to construct a well-formed batch request body. It also requires you to configure the service to support OData batching.

For example, the following GET request can be passed in a batch request as demonstrated further below:

GET: http://ServiceRoot/Movies?$select=Id,Name,Classification,RunningTime&$filter=contains(Name, 'li')&$orderby=Name desc

Wrapped in a batch request…

POST: http://ServiceRoot/$batch

Content-Type: multipart/mixed;boundary=changeset_6c67825c-8938-4f11-af6b-a25861ee53cc

Request Body:

--batch_d3bcb804-ee77-4921-9a45-761f98d32029

Content-Type: multipart/mixed; boundary=changeset_6c67825c-8938-4f11-af6b-a25861ee53cc

--changeset_6c67825c-8938-4f11-af6b-a25861ee53cc
Content-Type: application/http
Content-Transfer-Encoding: binary

GET http://ServiceRoot/Movies?$select=Id,Name,Classification,RunningTime&$filter=contains(Name,%20%27li%27)&$orderby=Name%20desc HTTP/1.1
OData-Version: 4.0
OData-MaxVersion: 4.0
Accept: application/json;odata.metadata=minimal
Accept-Charset: UTF-8
User-Agent: PostmanRuntime/7.24.1

--changeset_6c67825c-8938-4f11-af6b-a25861ee53cc--
--batch_d3bcb804-ee77-4921-9a45-761f98d32029--

Response Body:

--batchresponse_656768b7-d2d6-49fb-8251-303ec59cc42f
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 200 OK
Content-Type: application/json; odata.metadata=minimal; odata.streaming=true; charset=utf-8
OData-Version: 4.0

{"@odata.context":"http://ServiceRoot/$metadata#Movies(Id,Name,Classification,RunningTime)","value":[{"Id":11,"Name":"The Equalizer","Classification":"R","RunningTime":132},{"Id":9,"Name":"Dolittle","Classification":"PG","RunningTime":101}]}
--batchresponse_656768b7-d2d6-49fb-8251-303ec59cc42f--

In OData Web API 7.5, we introduced an alternative and relatively easier way of achieving the same goal. You can now pass the query options part of the URL in the request body to an endpoint comprising of the resource path appended with /$query – using the POST verb instead of GET.

Based on our example above, you can now do it this way:

POST: http://ServiceRoot/Movies/$query

Content-Type: text/plain

Request Body:

$select=Id,Name,Classification,RunningTime&$filter=contains(Name,%20%27li%27)&$orderby=Name%20desc

Response Body:

{
    "@odata.context": "http://ServiceRoot/$metadata#Movies(Id,Name,Classification,RunningTime)",
    "value": [
        {
            "Id": 11,
            "Name": "The Equalizer",
            "Classification": "R",
            "RunningTime": 132
        },
        {
            "Id": 9,
            "Name": "Dolittle",
            "Classification": "PG",
            "RunningTime": 101
        }
    ]
}

The OData specification prescribes that the query options specified in the request body and those specified in the request URL are processed together. Whereas a query string on a POST request is not a common thing, this feature supports that option. Here’s how the request would look like:

POST: http://ServiceRoot/Movies/$query?$filter=contains(Name,%20%27li%27)

Content-Type: text/plain

Request Body:

$select=Id,Name,Classification,RunningTime$orderby=Name%20desc

Specifically, note that /$query should be preceded by the resource path. The resource path is /Movies in the above example. This is the one way that the /$query endpoint differs from the /$batch endpoint.

Feature extensibility

We support the ability to extend this feature by providing means for the developer to plug in their own query options parser.

Your first step towards accomplishing that is implementing the IODataQueryOptionsParser interface. This interface contains two methods that you need to implement:

  • bool CanParse(HttpRequest) (or bool CanParse(HttpRequestMessage) in .NET FX)
  • Task<string> ParseAsync(Stream)

The logic to analyze the request to determine if the parser is capable of processing the content goes into CanParse(HttpRequest) method. Your logic could inspect headers, e.g. Content-Type or any other thing in the request to evaluate whether the conditions are right for the parser to process the request content.

The parsing logic goes into the ParseAsync(Stream) method. The return value for the method is a string containing the query options part of the URL.

Implementing a custom query options parser

In this section, we look at how one can implement and plug in their own parser. We implement a simple parser that reads and extracts query options from a POST with text/xml content.

public class TextXmlODataQueryOptionsParser : IODataQueryOptionsParser
{
    private static MediaTypeHeaderValue SupportedMediaType =
            MediaTypeHeaderValue.Parse("text/xml");

    public bool CanParse(Microsoft.AspNetCore.Http.HttpRequest request)
    {
        return request.ContentType?.StartsWith(SupportedMediaType.MediaType,
            StringComparison.Ordinal) == true ? true : false;
    }

    public async Task<string> ParseAsync(Stream requestStream)
    {
        using (var reader = new StreamReader(
                requestStream,
                encoding: Encoding.UTF8,
                detectEncodingFromByteOrderMarks: false,
                bufferSize: 1024,
                leaveOpen: true))
        {
            var content = await reader.ReadToEndAsync();
            var document = XDocument.Parse(content);
            var queryOptions = document.Descendants("QueryOption").Select(d =>
            new {
                Option = d.Attribute("Option").Value,
                d.Attribute("Value").Value
            });

            return string.Join("&", queryOptions.Select(d => d.Option + "=" + d.Value));
        }
    }
}

Let us go ahead and plug in this parser into the dependency injection container.

var queryOptionsParsers = ODataQueryOptionsParserFactory.Create();
queryOptionsParsers.Insert(0, new TextXmlODataQueryOptionsParser());

// ...
app.UseMvc(routeBuilder =>
{
    // ...
    routeBuilder.MapODataServiceRoute(
    "ROUTENAME",
    "ROUTEPREFIX",
    configureAction: containerBuilder => containerBuilder
        .AddService(Microsoft.OData.ServiceLifetime.Singleton,
            typeof(IEdmModel),
            _ => modelBuilder.GetEdmModel()) // YOUR EDM MODEL HERE
        .AddService(Microsoft.OData.ServiceLifetime.Singleton,
            typeof(IODataPathHandler),
            _ => new DefaultODataPathHandler())
        .AddService(Microsoft.OData.ServiceLifetime.Singleton,
            typeof(IEnumerable<IODataRoutingConvention>),
            _ => ODataRoutingConventions.CreateDefault())
        .AddService(Microsoft.OData.ServiceLifetime.Singleton,
            typeof(IEnumerable<IODataQueryOptionsParser>),
            _ => queryOptionsParsers)
    );
});

Note: Inserting the new parser at index 0 elevates it above any other that could potentially handle the request.

This parser can process the following request:

POST: http://ServiceRoot/Movies/$query

Content-Type: text/xml

Request Body:

<QueryOptions>
<QueryOption Option="$select" Value="Id,Name,Classification,RunningTime"/>
<QueryOption Option="$filter" Value="contains(Name,'li')"/>
<QueryOption Option="$orderby" Value="Name desc"/>
</QueryOptions>

Important notes

  1. The OData specification prescribes that the same system query option must not be specified more than once. For this reason, be careful not to repeat the same query option on the request URL, or on the request body, or both considered together. The current behaviour when a query option is repeated is a 404 response.
  2. Nothing changes in terms of how you implement controllers. You will still implement a Get controller action decorated with EnableQueryAttribute or taking a ODataQueryOptions<TEntity> parameter.

Summary

It is my hope that this feature will enrich your experience when dealing with long URLs situations.

If you have any questions, comments, concerns, or if you run into any issues using this feature, feel free to reach out on this blog post or report the issue on GitHub repo for OData Web API. We are more than happy to listen to your feedback and respond to your concerns.

John Gathogo Senior Software Engineer

Read next

12 comments

Comments are closed. Login to edit/delete your existing comments

  • David Taylor 0

    Thanks John. Can you please confirm if this custom query options parsing feature is only relevant if you are including parts of your query in the request body (HTTP POST)? Or is it a more general way I can intercept and rewrite a query (via the returned string) for a standard HTTP GET OData request?

    I have one other question about what validation you are doing if I intercept the query this way (assuming it supports a standard HTTP GET request). I have been waiting the last 1-2 years for your team to implement the OData $search query option feature. Can I now intercept and rewrite a query containing the $search options, or will your OData framework throw an invalid format exception because your team’s code does not yet recognise $search as a valid option?

    Thanks for any insight you can provide? I just want to understand if I should spend my time investigating this further to solve the lack of $search support? Note: I am trying to get an .NET 5 OData backend that works correctly against the standard Salesforce OData (Salesforce Connect) feature which supports $search.
    Regard, David

    • John GathogoMicrosoft employee 0

      > Can you please confirm if this custom query options parsing feature is only relevant if you are including parts of your query in the request body (HTTP POST)? Or is it a more general way I can intercept and rewrite a query (via the returned string) for a standard HTTP GET OData request?

      I’m not 100% sure I understand what you’re asking here but what the request body should contain is determined by the logic you write in your parser to process the request. What is important is that the parser outputs a query that would be valid in a typical OData GET request.

      For example, you could decide that the users of your service supply the following in the request body

      columns=FirstName,LastName,DoB
where=Title eq 'SE'
sort=DoB desc

      that the parser parses into

      $filter=Title eq 'SE'&$orderby=DoB desc&$select=FirstName,LastName,DoB

      > Can I now intercept and rewrite a query containing the $search options, or will your OData framework throw an invalid format exception because your team’s code does not yet recognise $search as a valid option?

      If you’re asking if you can have a $search option in the request body that your parser can rewrite into a query containing options supported in a standard OData GET request then the answer is yes.

      Kindly try to experiment with the feature and see if it meets your requirements. We’d be glad to get your feedback. If you identify something that you think we should consider, raise an issue on our repo.
      If you’re in a position to, you’re welcome to make a contribution w.r.t. support for $search or any other feature.

      • David Taylor 0

        Hi John,

        > If you’re asking if you can have a $search option in the request body that your parser can rewrite into a query containing options supported in a standard OData GET request then the answer is yes.

        That’s not quite what I was asking. I was asking if I can have a $search option in the “query string via a standard HTTP GET request” (I don’t want to put anything in the request body). Just to be clear, I have absolutely no control over the querystring (Salesforce does whatever salesforce does as per OData v4 query, but it uses $search which is not yet supported by .NET OData, even though it is a standard part of the specification).

        • John GathogoMicrosoft employee 0

          You still can’t have a `$search` on the query string since it’s not supported. The conversation on this issue might partly explain why it’s still pending but I hope we can get around to confronting the issues soon and implementing it to address the feature gap.

    • nAviD zehi 1

      Hi, Thanks John for your great article.
      I have tried this and I keep getting 404 when I add $query to my URL(example: localhost:1234/odata/weatherforecast/APost$query). Am I missing something. Should I enable $query somehow ?

      • John GathogoMicrosoft employee 0

        Hi @nAviD. Kindly confirm that your request is a POST. In addition, assuming APost is an entity set, the URL should end with /APost/$query. Otherwise, share a repro I can look at and advise…

    • anonymous 0

      this comment has been deleted.

    • anonymous 0

      this comment has been deleted.

      • Prasad Phate (Tata Consultancy Services Ltd)Microsoft employee 0

        test comment

        • anonymous 0

          this comment has been deleted.

        • Prasad Phate (Tata Consultancy Services Ltd)Microsoft employee 0

          test comment

  • Shreyashi Khare (Tata Consultancy Services Ltd)Microsoft employee 0

    test

Feedback usabilla icon