{"id":7010,"date":"2023-11-15T08:00:38","date_gmt":"2023-11-15T16:00:38","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/cosmosdb\/?p=7010"},"modified":"2024-06-13T14:02:14","modified_gmt":"2024-06-13T21:02:14","slug":"introducing-vector-search-in-azure-managed-instance-for-apache-cassandra","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cosmosdb\/introducing-vector-search-in-azure-managed-instance-for-apache-cassandra\/","title":{"rendered":"Introducing Vector Database in Azure Managed Instance for Apache Cassandra!"},"content":{"rendered":"

We are delighted to announce Cassandra 5.0<\/a> in public preview<\/strong> for Azure Managed Instance for Apache Cassandra<\/a>, with vector search<\/a> capabilities enabled!<\/p>\n

 <\/p>\n

Vector Database in Cassandra 5.0<\/span><\/h2>\n

In Cassandra 5.0, the integrated vector database is backed by storage-attached indexes that represent data as vectors in a multi-dimensional space. We use DiskANN<\/a>, a Microsoft Research-born algorithm that allows for adding new vectors to an index without the need for reindexing, which causes downtime in other vector stores. <\/span>DiskANN enables fast and accurate similarity-based searches, which can be used to build next-generation AI apps that use Azure OpenAI<\/a> and other LLMs.<\/span><\/p>\n

With the integrated vector database, your applications find similar items based on their data characteristics rather than exact matches on a property field. They<\/span> work by taking the vector representations (lists of numbers) of your data that you have created using an ML model, or an embeddings API such as Azure OpenAI Service Embeddings<\/a>\u00a0or\u00a0Hugging Face on Azure<\/a>. They then measure the distance between the data vectors and your query vector. The data vectors that are closest to your query vector are the ones that are found to be most similar semantically.<\/p>\n

<\/h2>\n

Create a Vector Index<\/span><\/h4>\n

In Cassandra, first create a keyspace and table which has the new vector data type. The second parameter represents the number of dimensions that the vector can hold. In this case, we’ll only add 2 dimensions. However, the dimensionality of your vectors may be different. For example, if you were using OpenAI Embeddings<\/a>, this would be set to 1536.<\/em><\/p>\n

CREATE KEYSPACE test <\/span><\/code>WITH replication = {'class': 'NetworkTopologyStrategy', 'dc1': '3'};\r\n<\/span>use test;\r\n<\/span>CREATE TABLE s (k int PRIMARY KEY, v vector<float, 2>);<\/span><\/code><\/pre>\n
Next, create an index on the vector field:<\/p>\n
CREATE CUSTOM INDEX ON s(v) USING 'StorageAttachedIndex';<\/code><\/pre>\n
<\/h2>\n
Add vectors to your table<\/span><\/h4>\n
To add vectors to your table, you can use the OpenAI Embeddings model to generate embeddings from the data<\/a>. In this example, we\u2019ll insert a few rows that contain sample embeddings:<\/p>\n
INSERT INTO s (k, v) VALUES (1, [1, 1.8]);\r\nINSERT INTO s (k, v) VALUES (1, [1, 2.1]);<\/code><\/pre>\n
<\/h2>\n
Perform a vector search<\/span><\/h4>\n
Finally, you can order by the vector column, with an Approximate Nearest Neighbour (ANN) search:<\/p>\n
SELECT k, v FROM s WHERE k = 1 ORDER BY v ANN OF [2, 2] LIMIT 4 ALLOW FILTERING;<\/code><\/pre>\n
 <\/p>\n
Generative AI Apps<\/span><\/h2>\n
Lets put this all together in an app that uses vector search in Azure Managed Instance for Apache Cassandra! To get you started, we’ve created a Java-based ChatGPT-like sample app<\/a> that integrates with Cassandra 5.0 using vector search. The sample allows you to load a file containing your own private contextual data, and uses Azure OpenAI features to create a chat experience.<\/p>\n
Included in the sample are:<\/p>\n
    \n
  1. Azure OpenAI embedding with vector dimensionality of 1536. Azure OpenAI will automatically create embeddings from the data loaded into the vector store.<\/li>\n
  2. Loading and searching the data from a vector store table in Cassandra 5.0, and performing a similarity search using Approximate Nearest Neighbour (ANN).<\/li>\n
  3. Generating prompts and marshalling chat history (in memory).<\/li>\n
  4. Chat completion using Azure OpenAI.<\/li>\n<\/ol>\n
     <\/p>\n
    How it works<\/h4>\n
    \"workflow.png\"<\/p>\n
    Follow the instructions in the repo to create a generative AI chat experience using vector search in Azure Managed Instance for Apache Cassandra! Once you’ve loaded your data, you can ask the app questions and it will answer based on the private contextual data you provided.<\/p>\n
    Be careful not to use data that’s too biased! \ud83d\ude42<\/p>\n
     <\/p>\n
    https:\/\/devblogs.microsoft.com\/cosmosdb\/wp-content\/uploads\/sites\/52\/2023\/11\/chatgpt-2.mp4<\/a><\/video><\/div><\/p>\n
     <\/p>\n
    Vector search at scale<\/span><\/h2>\n
    Apache Cassandra is a massively scalable database with transparent partitioning<\/a> that stores data across many independent, fault-tolerant nodes in a cluster. Vector searches in Cassandra 5.0 will perform much better if searches are directed towards vectors stored in a single partition.<\/p>\n
    In the app sample above, by default, each data file you load into the vector store will generate 1 partition of vectors in the table. If you load multiple files, this will result in cross-partition key queries when the app attempts to perform a vector search against your contextual data.<\/p>\n
    For large-scale apps that leverage vector search, you should design your approach so that, as much as possible, similarity searches are scoped to a single partition or a small number of partitions.<\/p>\n
    \"Image<\/a><\/p>\n
    Getting started<\/span><\/h2>\n