Announcing General Availability of Azure Cosmos DB integrated cache

Justine Cocchi

August 31st, 20220 3

Reduce costs and latency for read–heavy workloads with Azure Cosmos DB integrated cache, an in-memory cache that’s now generally available. In addition to increasing speed and decreasing costs, adding an integrated cache simplifies your architecture and application complexity, allowing you to focus on building your business logic. The integrated cache is built into the Azure Cosmos DB dedicated gateway and can be utilized with minimal changes to your application. This allows you to get all the benefits of a traditional cache without any of the application overhead.

Save costs on database reads

Integrated cache can reduce database costs for many workloads. For example, it will cost 0 request units (RUs) to serve a read from the integrated cache if no cache refresh is required. This is true for point reads or queries, regardless of the complexity of the query or the size of data returned. This is useful for:

Read-heavy workloads with relatively static data. If data is read more often than it is updated, there is potential benefit from caching.
Workloads that perform the same point read or query multiple times within a short period. Point reads of the same document and queries with the same text, including parameters, that are executed multiple times within the staleness window you set will cost 0 RUs!
Workloads that need predictable, low latency for repeated point reads and queries.
Workloads that have expensive reads, either due to large document sizes or complex queries.
Workloads that have a hot partition due to repeated reads within one or more physical partitions.

New integrated cache features

We added new features since the public preview including:

Network isolation capabilities to help you keep your data secure. This includes the ability to provision a dedicated gateway on Azure Cosmos DB accounts configured to use a Virtual Network, IP filters, or Private Link.
New supported regions.
Improved metrics to help you monitor your cache usage.
The ability to set the MaxIntegratedCacheStaleness value with more of our SDKs! See examples for each SDK in our documentation.

Get started with integrated cache

First you need to provision a dedicated gateway. Once deployed, the dedicated gateway functions similar to the Azure Cosmos DB standard gateway for routing requests, and also contains the integrated cache.

To use the dedicated gateway and integrated cache, ensure three things are true in your application.

Your application should be configured to use the dedicated gateway connection string. The endpoint will replace “documents.azure.com” with “sqlx.cosmos.azure.com” and the rest remains the same. Here is an example of finding the dedicated gateway connection string in the Portal.
Your application should be in gateway mode. Direct mode is the default for the .NET and Java SDKs so be sure to set gateway mode explicitly when creating your client. The Python and Node.js SDKs always use gateway mode.
A request should have either Session or Eventual consistency to use the integrated cache. You can use the dedicated gateway with other consistency modes, but requests with the other modes will bypass the integrated cache. These requests cost RUs even if they would have been a cache hit. You can configure consistency at the account, client, or request level. The default account consistency setting doesn’t apply for the Python SDK, so you need to set it for each request.