We’re excited to announce general availability of DiskANN and Filtered Vector Search on Azure Cosmos DB for MongoDB (vCore), starting with M30 cluster tiers and above. You can now use these features in production to store and query vector embeddings directly alongside your operational data—efficiently and in one integrated vector database. 

DiskANN Indexing: Scaling Vector Search to Massive Datasets  

 Efficient vector indexing is critical for AI applications that process large-scale datasets. While HNSW and IVF each offer advantages, we recommend DiskANN for datasets with more than 1 million documents. DiskANN is designed to take full advantage of SSDs, bypassing the memory limitations that can hinder other indexing methods at scale. It delivers strong performance with high recall, helping you accurately retrieve the most relevant neighbors—even from massive data pools. To use DiskANN, simply specify “kind”: “vector-diskann” when creating your index. 

{ 
   "createIndexes": "testCollection", 
   "indexes": [ 
       { 
           "name": "DiskANNVectorIndex", 
           "key": { 
               "contentVector": "cosmosSearch" 
           }, 
           "cosmosSearchOptions": { 
               "kind": "vector-diskann", 
               "dimensions": 3, 
               "similarity": "COS", 
               "maxDegree": 32, 
               "lBuild": 64 
           } 
       } 
   ] 
}

Optimizing with Product Quantization (PQ) 

 When working with extremely large datasets or high-dimensional vectors, you can further optimize DiskANN by enabling Product Quantization (PQ). PQ compresses the vector index, reducing storage requirements and potentially speeding up search operations. However, this compression may come with a trade-off in search accuracy. To enable PQ, specify “compression”: “pq” when creating your DiskANN index. 

db.runCommand( 
{ 
   "createIndexes": "your_vector_collection", 
   "indexes": [ 
       { 
           "key": { "v": "cosmosSearch" }, 
           "name": "diskann_pq_index", 
           "cosmosSearchOptions": { 
               "kind": "vector-diskann", 
               "similarity": "COS", 
               "dimensions": 1536, // Max 16,000 dimensions  
               "compression": "pq", 
               "pqCompressedDims": 96, // Compressed dimensions 
               "pqSampleSize": 2000    // Training samples 
           } 
       } 
   ] 
} )  

 Using DiskANN with PQ offers a major advantage: support for vectors with dimensions up to 16,000. This expanded limit lets you work with embeddings from the latest and most complex AI models. 

To help offset any potential accuracy loss from PQ compression, you can apply the oversampling parameter in your search queries. 

db.your_vector_collection.aggregate([ 
   { 
       $search: { 
           "cosmosSearch": { 
               "vector": [0.1, 0.5, 0.9, ...], // Your query vector 
               "path": "v", 
               "k": 10, // Number of results to return 
               "oversampling": 2.0 // Retrieve 2 * 10 = 20 candidates for reranking 
           }, 
           "returnStoredSource": true // Optional: return the original document 
       } 
   } 
])  

By fetching extra candidates from the compressed index and then re-evaluating them, oversampling helps to restore accuracy and provide a more precise set of top k results. 

Filtered Vector Search: Adding Precision to Relevance 

 In real-world applications, finding semantically similar items is just one part of the challenge. You often need to refine results based on attributes like category, location, or status. 

Filtered Vector Search lets you apply standard MongoDB query filters—such as $eq, $in, or $geoWithin—before running a vector search. To use these filters, make sure you create a regular index on the field you want to filter by. 

pipeline = [ 
   { 
       "$search": { 
           "cosmosSearch": { 
               "path": "contentVector", 
               "vector": query_vector, # Assuming query_vector is defined 
               "k": 5, 
               "filter": { 
                   "$and": [ 
                       {"is_open": {"$eq": 1}}, # Example filter 1 
                       {"location": {"$geoWithin": {"$centerSphere": [[-119.7192861804, 34.4102485028], 100 / 3963.2]}}} # Example filter 2 
                   ] 
               } 
           } 
       } 
   } 
] 

Ready to Build with Vector Search 

With DiskANN and Filtered Vector Search now generally available on Azure Cosmos DB for MongoDB (vCore), you can build AI-powered applications that combine semantic relevance with operational filtering at scale. Whether you’re handling millions of high-dimensional vectors or narrowing results with precise filters, these capabilities bring the performance, flexibility, and integration needed for modern intelligent workloads. Explore what’s possible, optimize your search, and unlock new value from your data. 

Leave a review 

Tell us about your Azure Cosmos DB experience! Leave a review on PeerSpot and we’ll gift you $50. Get started here. 

About Azure Cosmos DB 

Azure Cosmos DB is a fully managed and serverless NoSQL and vector database for modern app development, including AI applications. With its SLA-backed speed and availability as well as instant dynamic scalability, it is ideal for real-time NoSQL and MongoDB applications that require high performance and distributed computing over massive volumes of NoSQL and vector data. 

To stay in the loop on Azure Cosmos DB updates, follow us on X, YouTube, and LinkedIn.