{"id":10241,"date":"2025-05-21T11:00:13","date_gmt":"2025-05-21T18:00:13","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/cosmosdb\/?p=10241"},"modified":"2025-05-14T08:11:57","modified_gmt":"2025-05-14T15:11:57","slug":"diskann-in-azure-cosmos-db-for-mongodb","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cosmosdb\/diskann-in-azure-cosmos-db-for-mongodb\/","title":{"rendered":"DiskANN and Filtered Vector Search are Now Generally Available in Azure Cosmos DB for MongoDB (vCore)"},"content":{"rendered":"

We\u2019re<\/span> excited to announce <\/span>general<\/span> availability of <\/span>DiskANN<\/span><\/a> and Filtered Vector Search on Azure Cosmos DB for MongoDB (<\/span>vCore<\/span>), starting with M30 cluster tiers and above. You can now use these features in production to store and query vector embeddings directly alongside your operational data\u2014efficiently and in one integrated vector database.<\/span><\/span>\u00a0<\/span><\/p>\n

\"DiskANN<\/a><\/p>\n

DiskANN<\/span> Indexing: Scaling Vector Search to Massive Datasets <\/span>\u00a0<\/span><\/h2>\n

\u00a0Efficient vector indexing is critical for AI applications that process large-scale datasets. While HNSW and IVF each offer advantages, we recommend DiskANN<\/a> for datasets with more than 1 million documents. DiskANN is designed to take full advantage of SSDs, bypassing the memory limitations that can hinder other indexing methods at scale. It delivers strong performance with high recall, helping you accurately retrieve the most relevant neighbors\u2014even from massive data pools. To use DiskANN, simply specify <\/span>“kind”: “vector-diskann”<\/strong> when creating your index.<\/span>\u00a0<\/span><\/p>\n

{\u00a0\r\n\u00a0\u00a0\u00a0\"createIndexes\": \"testCollection\",\u00a0\r\n\u00a0\u00a0\u00a0\"indexes\": [\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0{\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"name\": \"DiskANNVectorIndex\",\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"key\": {\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"contentVector\": \"cosmosSearch\"\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0},\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"cosmosSearchOptions\": {\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"kind\": \"vector-diskann\",\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"dimensions\": 3,\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"similarity\": \"COS\",\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"maxDegree\": 32,\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"lBuild\": 64\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\u00a0\r\n\u00a0\u00a0\u00a0]\u00a0\r\n}<\/code><\/pre>\n
Optimizing<\/span> with Product Quantization (PQ)<\/span>\u00a0<\/span><\/h2>\n
\u00a0When working with extremely large datasets or high-dimensional vectors, you can further optimize DiskANN by enabling Product Quantization (PQ). PQ compresses the vector index, reducing storage requirements and potentially speeding up search operations. However, this compression may come with a trade-off in search accuracy. To enable PQ, specify <\/span>“compression”: “pq”<\/span> when creating your DiskANN index.<\/span>\u00a0<\/span><\/p>\n
db.runCommand(\u00a0\r\n{\u00a0\r\n\u00a0\u00a0\u00a0\"createIndexes\": \"your_vector_collection\",\u00a0\r\n\u00a0\u00a0\u00a0\"indexes\": [\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0{\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"key\": { \"v\": \"cosmosSearch\" },\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"name\": \"diskann_pq_index\",\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"cosmosSearchOptions\": {\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"kind\": \"vector-diskann\",\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"similarity\": \"COS\",\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"dimensions\": 1536, \/\/ Max 16,000 dimensions \u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"compression\": \"pq\",\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"pqCompressedDims\": 96, \/\/ Compressed dimensions\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"pqSampleSize\": 2000 \u00a0\u00a0\u00a0\/\/ Training samples\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\u00a0\r\n\u00a0\u00a0\u00a0]\u00a0\r\n} )\u00a0<\/code>\u00a0<\/span><\/pre>\n
\u00a0Using DiskANN with PQ offers a major advantage: support for vectors with dimensions up to 16,000. This expanded limit lets you work with embeddings from the latest and most complex AI models.<\/span>\u00a0<\/span><\/p>\n
To help offset any potential accuracy loss from PQ compression, you can apply the oversampling parameter in your search queries.<\/span>\u00a0<\/span><\/p>\n
db.your_vector_collection.aggregate([\u00a0\r\n\u00a0\u00a0\u00a0{\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0$search: {\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"cosmosSearch\": {\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"vector\": [0.1, 0.5, 0.9, ...], \/\/ Your query vector\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"path\": \"v\",\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"k\": 10, \/\/ Number of results to return\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"oversampling\": 2.0 \/\/ Retrieve 2 * 10 = 20 candidates for reranking\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0},\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"returnStoredSource\": true \/\/ Optional: return the original document\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\u00a0\r\n\u00a0\u00a0\u00a0}\u00a0\r\n])\u00a0<\/code>\u00a0<\/span><\/pre>\n
By fetching extra candidates from the compressed index and then re-evaluating them, oversampling helps to restore accuracy and provide a more precise set of top k results.<\/span>\u00a0<\/span><\/p>\n
Filtered Vector Search: Adding Precision to Relevance<\/span>\u00a0<\/span><\/h2>\n
\u00a0In real-world applications, finding semantically similar items is just one part of the challenge. You often need to refine results based on attributes like category, location, or status.<\/span>\u00a0<\/span><\/p>\n
Filtered Vector Search lets you apply standard MongoDB query filters\u2014such as <\/span>$eq<\/em>, <\/span>$in<\/em>, or <\/span>$geoWithin<\/em>\u2014before running a vector search. To use these filters, make sure you create a regular index on the field you want to filter by.<\/span>\u00a0<\/span><\/p>\n
pipeline = [\u00a0\r\n\u00a0\u00a0\u00a0{\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"$search\": {\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"cosmosSearch\": {\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"path\": \"contentVector\",\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"vector\": query_vector, # Assuming query_vector is defined\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"k\": 5,\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"filter\": {\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"$and\": [\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0{\"is_open\": {\"$eq\": 1}}, # Example filter 1\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0{\"location\": {\"$geoWithin\": {\"$centerSphere\": [[-119.7192861804, 34.4102485028], 100 \/ 3963.2]}}} # Example filter 2\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0]\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\u00a0\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0}\u00a0\r\n\u00a0\u00a0\u00a0}\u00a0\r\n] <\/code><\/pre>\n
Ready to Build with Vector Search<\/span>\u00a0<\/span><\/h2>\n
With DiskANN and Filtered Vector Search<\/a> now generally available on Azure Cosmos DB for MongoDB (vCore), you can build AI-powered applications that combine semantic relevance with operational filtering at scale. Whether you’re handling millions of high-dimensional vectors or narrowing results with precise filters, these capabilities bring the performance, flexibility, and integration needed for modern intelligent workloads. Explore what\u2019s possible, optimize your search, and unlock new value from your data.<\/span>\u00a0<\/span><\/p>\n
Leave a review<\/span><\/b>\u00a0<\/span><\/h3>\n
Tell us about your Azure Cosmos DB experience! Leave a review on PeerSpot and we\u2019ll gift you $50. <\/span>Get started here<\/span><\/a>.<\/span>\u00a0<\/span><\/p>\n
About Azure Cosmos DB<\/span><\/b>\u00a0<\/span><\/h3>\n
Azure Cosmos DB is a fully managed and serverless NoSQL and vector database for modern app development, including AI applications. With its SLA-backed speed and availability as well as instant dynamic scalability, it is ideal for real-time NoSQL and MongoDB applications that require high performance and distributed computing over massive volumes of NoSQL and vector data.<\/span>\u00a0<\/span><\/p>\n
To stay in the loop on Azure Cosmos DB updates, follow us on <\/span>X<\/span><\/a>, <\/span>YouTube<\/span><\/a>, and <\/span>LinkedIn<\/span><\/a>.<\/span>\u00a0<\/span><\/p>\n
Get Started with DiskANN<\/a><\/div><\/p>\n","protected":false},"excerpt":{"rendered":"
We\u2019re excited to announce general availability of DiskANN and Filtered Vector Search on Azure Cosmos DB for MongoDB (vCore), starting with M30 cluster tiers and above. You can now use these features in production to store and query vector embeddings directly alongside your operational data\u2014efficiently and in one integrated vector database.\u00a0 DiskANN Indexing: Scaling Vector […]<\/p>\n","protected":false},"author":125132,"featured_media":10243,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[15],"tags":[],"class_list":["post-10241","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-mongodb-api"],"acf":[],"blog_post_summary":"
We\u2019re excited to announce general availability of DiskANN and Filtered Vector Search on Azure Cosmos DB for MongoDB (vCore), starting with M30 cluster tiers and above. You can now use these features in production to store and query vector embeddings directly alongside your operational data\u2014efficiently and in one integrated vector database.\u00a0 DiskANN Indexing: Scaling Vector […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts\/10241","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/users\/125132"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/comments?post=10241"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/posts\/10241\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/media\/10243"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/media?parent=10241"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/categories?post=10241"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-json\/wp\/v2\/tags?post=10241"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}