Introducing priority-based execution in Azure Cosmos DB (preview)

Richa Gaur

The problem  

Azure Cosmos DB is a fast and flexible database service that can handle massive amounts of data and scale elastically. However, sometimes you may encounter situations where your application sends more requests than the configured throughput (RU/s) of your container. In such cases, Azure Cosmos DB will throttle some of the requests and return a 429 status code with a retry-after header. 

This situation can be frustrating and can lead to degradation of application performance and even revenue loss.  

The Solution 

What if there was a way to overcome this situation without incurring the extra cost of provisioning higher RU/sec? 

Priority-based execution is a capability which allows users to specify priority for the request sent to Azure Cosmos DB. Based on the priority specified by the user, if there are more requests than the configured RU/s in a second, then Azure Cosmos DB will throttle low priority requests to allow high priority requests to execute. 

This capability allows a user to perform more important tasks while delaying less important tasks when there are a higher number of requests than what a container with configured RU/s can handle at a given time. Less important tasks will be continuously retried by any client using an SDK based on the retry policy configured. 

How does priority-based execution work? 

Let’s take an example of an e-commerce application that uses Azure Cosmos DB as its backend. Suppose the application has two types of requests: one for displaying product information to users and another for updating the product catalog.  

Let’s say you have provisioned 400 RU/sec on the product container. During peak hours, 400 RPS requests come in for displaying product information, while 150 RPS requests come in for catalog updates. 

For simplicity, let’s assume that each request consumes 1 RU/sec for each transaction. We can observe how Azure Cosmos DB behaves in such a scenario using the metrics below. 

As you can observe from the above metrics, both product information requests and catalog update requests are throttled when the consumption of the container is above 400 RU/s. This is fair since a provisioned resource can’t accept load beyond its configured capacity. However, for customers these kinds of scenarios may not be favorable. When product information display requests are not completed, customers can’t see product information and therefore can’t make purchases.  

Seeing it in operation  

Let’s see how priority-based execution can solve this problem. 

The product display requests are more critical than the catalog updates, so logically, they should be given higher priority. The application can use the following code snippet to set the priority of each request using the SDK. 

Using Mircosoft.Azure.Cosmos.PartitionKey; 
Using Mircosoft.Azure.Cosmos.PriorityLevel; 

//update products catalog  
RequestOptions catalogRequestOptions = new ItemRequestOptions{PriorityLevel = PriorityLevel.Low}; 

PartitionKey pk = new PartitionKey(“productId1”); 
ItemResponse<Product> catalogResponse = await this.container.CreateItemAsync<Product>(product1, pk, requestOptions); 

//Display product information to user 
RequestOptions getProductRequestOptions = new ItemRequestOptions{PriorityLevel = PriorityLevel.High}; 

string id = “productId2”; 
PartitionKey pk = new PartitionKey(id); 

ItemResponse<Product> productResponse = await this.container.ReadItemAsync< Product>(id, pk, getProductRequestOptions); 

Let’s observe the behavior of Azure Cosmos DB for a similar workload with priority-based execution enabled using the metrics below. 

With priority-based execution, when the total consumption of the container exceeds the configured RU/s, Azure Cosmos DB first throttles low-priority requests, allowing high-priority requests to execute in a high load situation.  This helps users execute important requests and improve the customer experience without overprovisioning RU/s in the container and incurring extra costs. 

Key Benefits 

• Optimize your application performance and user experience by ensuring that your high-priority requests get executed while your low-priority requests wait for their turn during peak load.
• Save costs by avoiding over-provisioning RU/s for your container to avoid throttling of high priority requests.

Limitations  

There are a few limitations to this feature. Priority-based throttling is a best effort scenario, and there are no SLA’s defined for the performance of the feature. 

Use-cases 

You can use priority-based execution when your application has different priorities for workloads running on the same container. For example, 

• Prioritizing read, write, or query operations.  
• Prioritizing user actions vs background operations like  

• • Stored procedures 
  • Data ingestion/migration 

Final thoughts 

Priority-based execution is an excellent addition to the Azure Cosmos DB service, providing users with a way to optimize their application performance and user experience while minimizing the cost of overprovisioning RU/s.  

If you have any questions or feedback, please leave them in the comments section below. Happy coding! 

Next Steps 

Get started with priority-based execution by following the steps outlined below. 

1. Start by filling out this nomination form. After submitting, we will enable the feature on the accounts you listed and contact you to let you know it’s ready for use. 
2. Customers accepted for the preview will need to download these SDK versions to use the feature. 
   • .NET SDK: Azure Cosmos DB dotnetv3 SDK- 3.33.0-preview 
   • Java SDK: Azure Cosmos DB Java SDK – 4.45.0 
3. Send us your feedback, comments, or questions using this Github repository. 

Azure Cosmos DB is a fully managed NoSQL and relational database for modern app development with SLA-backed speed and availability, automatic and instant scalability, and support for open source PostgreSQL, MongoDB and Apache Cassandra. Try Azure Cosmos DB for free here. To stay in the loop on Azure Cosmos DB updates, follow us on Twitter, YouTube, and LinkedIn. 

Richa Gaur Program Manager , Cosmos DB

Follow