The query options part of an OData URL can be quite long, potentially exceeding the maximum length of URLs supported by components involved in transmitting or processing the request. HTTP does not impose any limits on the length of a URL, however, many hosting environments (including IIS) impose limitations which may restrict the client’s ability to request the exact set of data that they are interested in. These limitations would especially pose a big challenge for clients using explicit select statements for example.
Background
One way that you can go round these limitations is by wrapping the GET request in a batch request. This approach requires you to be conversant with how to construct a well-formed batch request body. It also requires you to configure the service to support OData batching.
For example, the following GET request can be passed in a batch request as demonstrated further below:
GET: http://ServiceRoot/Movies?$select=Id,Name,Classification,RunningTime&$filter=contains(Name, 'li')&$orderby=Name desc
Wrapped in a batch request…
POST: http://ServiceRoot/$batch
Content-Type: multipart/mixed;boundary=changeset_6c67825c-8938-4f11-af6b-a25861ee53cc
Request Body:
--batch_d3bcb804-ee77-4921-9a45-761f98d32029 Content-Type: multipart/mixed; boundary=changeset_6c67825c-8938-4f11-af6b-a25861ee53cc --changeset_6c67825c-8938-4f11-af6b-a25861ee53cc Content-Type: application/http Content-Transfer-Encoding: binary GET http://ServiceRoot/Movies?$select=Id,Name,Classification,RunningTime&$filter=contains(Name,%20%27li%27)&$orderby=Name%20desc HTTP/1.1 OData-Version: 4.0 OData-MaxVersion: 4.0 Accept: application/json;odata.metadata=minimal Accept-Charset: UTF-8 User-Agent: PostmanRuntime/7.24.1 --changeset_6c67825c-8938-4f11-af6b-a25861ee53cc-- --batch_d3bcb804-ee77-4921-9a45-761f98d32029--
Response Body:
--batchresponse_656768b7-d2d6-49fb-8251-303ec59cc42f
Content-Type: application/http
Content-Transfer-Encoding: binary
HTTP/1.1 200 OK
Content-Type: application/json; odata.metadata=minimal; odata.streaming=true; charset=utf-8
OData-Version: 4.0
{"@odata.context":"http://ServiceRoot/$metadata#Movies(Id,Name,Classification,RunningTime)","value":[{"Id":11,"Name":"The Equalizer","Classification":"R","RunningTime":132},{"Id":9,"Name":"Dolittle","Classification":"PG","RunningTime":101}]}
--batchresponse_656768b7-d2d6-49fb-8251-303ec59cc42f--
In OData Web API 7.5, we introduced an alternative and relatively easier way of achieving the same goal. You can now pass the query options part of the URL in the request body to an endpoint comprising of the resource path appended with /$query – using the POST verb instead of GET.
Based on our example above, you can now do it this way:
POST: http://ServiceRoot/Movies/$query
Content-Type: text/plain
Request Body:
$select=Id,Name,Classification,RunningTime&$filter=contains(Name,%20%27li%27)&$orderby=Name%20desc
Response Body:
{
"@odata.context": "http://ServiceRoot/$metadata#Movies(Id,Name,Classification,RunningTime)",
"value": [
{
"Id": 11,
"Name": "The Equalizer",
"Classification": "R",
"RunningTime": 132
},
{
"Id": 9,
"Name": "Dolittle",
"Classification": "PG",
"RunningTime": 101
}
]
}
The OData specification prescribes that the query options specified in the request body and those specified in the request URL are processed together. Whereas a query string on a POST request is not a common thing, this feature supports that option. Here’s how the request would look like:
POST: http://ServiceRoot/Movies/$query?$filter=contains(Name,%20%27li%27)
Content-Type: text/plain
Request Body:
$select=Id,Name,Classification,RunningTime$orderby=Name%20desc
Specifically, note that /$query should be preceded by the resource path. The resource path is /Movies in the above example. This is the one way that the /$query endpoint differs from the /$batch endpoint.
Feature extensibility
We support the ability to extend this feature by providing means for the developer to plug in their own query options parser.
Your first step towards accomplishing that is implementing the IODataQueryOptionsParser interface. This interface contains two methods that you need to implement:
bool CanParse(HttpRequest)(orbool CanParse(HttpRequestMessage) in .NET FX)Task<string> ParseAsync(Stream)
The logic to analyze the request to determine if the parser is capable of processing the content goes into CanParse(HttpRequest) method. Your logic could inspect headers, e.g. Content-Type or any other thing in the request to evaluate whether the conditions are right for the parser to process the request content.
The parsing logic goes into the ParseAsync(Stream) method. The return value for the method is a string containing the query options part of the URL.
Implementing a custom query options parser
In this section, we look at how one can implement and plug in their own parser. We implement a simple parser that reads and extracts query options from a POST with text/xml content.
public class TextXmlODataQueryOptionsParser : IODataQueryOptionsParser
{
private static MediaTypeHeaderValue SupportedMediaType =
MediaTypeHeaderValue.Parse("text/xml");
public bool CanParse(Microsoft.AspNetCore.Http.HttpRequest request)
{
return request.ContentType?.StartsWith(SupportedMediaType.MediaType,
StringComparison.Ordinal) == true ? true : false;
}
public async Task<string> ParseAsync(Stream requestStream)
{
using (var reader = new StreamReader(
requestStream,
encoding: Encoding.UTF8,
detectEncodingFromByteOrderMarks: false,
bufferSize: 1024,
leaveOpen: true))
{
var content = await reader.ReadToEndAsync();
var document = XDocument.Parse(content);
var queryOptions = document.Descendants("QueryOption").Select(d =>
new {
Option = d.Attribute("Option").Value,
d.Attribute("Value").Value
});
return string.Join("&", queryOptions.Select(d => d.Option + "=" + d.Value));
}
}
}
Let us go ahead and plug in this parser into the dependency injection container.
var queryOptionsParsers = ODataQueryOptionsParserFactory.Create();
queryOptionsParsers.Insert(0, new TextXmlODataQueryOptionsParser());
// ...
app.UseMvc(routeBuilder =>
{
// ...
routeBuilder.MapODataServiceRoute(
"ROUTENAME",
"ROUTEPREFIX",
configureAction: containerBuilder => containerBuilder
.AddService(Microsoft.OData.ServiceLifetime.Singleton,
typeof(IEdmModel),
_ => modelBuilder.GetEdmModel()) // YOUR EDM MODEL HERE
.AddService(Microsoft.OData.ServiceLifetime.Singleton,
typeof(IODataPathHandler),
_ => new DefaultODataPathHandler())
.AddService(Microsoft.OData.ServiceLifetime.Singleton,
typeof(IEnumerable<IODataRoutingConvention>),
_ => ODataRoutingConventions.CreateDefault())
.AddService(Microsoft.OData.ServiceLifetime.Singleton,
typeof(IEnumerable<IODataQueryOptionsParser>),
_ => queryOptionsParsers)
);
});
Note: Inserting the new parser at index 0 elevates it above any other that could potentially handle the request.
This parser can process the following request:
POST: http://ServiceRoot/Movies/$query
Content-Type: text/xml
Request Body:
<QueryOptions> <QueryOption Option="$select" Value="Id,Name,Classification,RunningTime"/> <QueryOption Option="$filter" Value="contains(Name,'li')"/> <QueryOption Option="$orderby" Value="Name desc"/> </QueryOptions>
Important notes
- The OData specification prescribes that the same system query option must not be specified more than once. For this reason, be careful not to repeat the same query option on the request URL, or on the request body, or both considered together. The current behaviour when a query option is repeated is a 404 response.
- Nothing changes in terms of how you implement controllers. You will still implement a
Getcontroller action decorated withEnableQueryAttributeor taking aODataQueryOptions<TEntity>parameter.
Summary
It is my hope that this feature will enrich your experience when dealing with long URLs situations.
If you have any questions, comments, concerns, or if you run into any issues using this feature, feel free to reach out on this blog post or report the issue on GitHub repo for OData Web API. We are more than happy to listen to your feedback and respond to your concerns.
test
Thanks John. Can you please confirm if this custom query options parsing feature is only relevant if you are including parts of your query in the request body (HTTP POST)? Or is it a more general way I can intercept and rewrite a query (via the returned string) for a standard HTTP GET OData request?
I have one other question about what validation you are doing if I intercept the query this...
this comment has been deleted.
test comment
test comment
this comment has been deleted.
this comment has been deleted.
Hi, Thanks John for your great article.
I have tried this and I keep getting 404 when I add $query to my URL(example: localhost:1234/odata/weatherforecast/APost$query). Am I missing something. Should I enable $query somehow ?
Hi @nAviD. Kindly confirm that your request is a POST. In addition, assuming APost is an entity set, the URL should end with /APost/$query. Otherwise, share a repro I can look at and advise…
> Can you please confirm if this custom query options parsing feature is only relevant if you are including parts of your query in the request body (HTTP POST)? Or is it a more general way I can intercept and rewrite a query (via the returned string) for a standard HTTP GET OData request?
I'm not 100% sure I understand what you're asking here but what the request body should contain is determined by the logic...
Hi John,
> If you’re asking if you can have a $search option in the request body that your parser can rewrite into a query containing options supported in a standard OData GET request then the answer is yes.
That's not quite what I was asking. I was asking if I can have a $search option in the "query string via a standard HTTP GET request" (I don't want to put anything in the request body)....
You still can’t have a `$search` on the query string since it’s not supported. The conversation on this issue might partly explain why it’s still pending but I hope we can get around to confronting the issues soon and implementing it to address the feature gap.