{"id":9205,"date":"2024-12-05T07:00:11","date_gmt":"2024-12-05T15:00:11","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/cosmosdb\/?p=9205"},"modified":"2024-12-05T13:25:54","modified_gmt":"2024-12-05T21:25:54","slug":"announcing-azure-cosmos-db-integration-with-spring-ai-and-langchain4j","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cosmosdb\/announcing-azure-cosmos-db-integration-with-spring-ai-and-langchain4j\/","title":{"rendered":"Announcing Azure Cosmos DB Integration with Spring AI and Langchain4J!"},"content":{"rendered":"

We\u2019re continuing to simplify AI app development by integrating Azure Cosmos DB\u2019s<\/strong> cost-effective and scalable vector search directly with Spring AI<\/strong><\/a> and LangChain4J<\/strong><\/a>! These frameworks empower Java developers to efficiently manage fast and accurate data retrieval with Azure Cosmos DB\u2019s scalability and efficient vector indexing and search<\/a> capabilities, allowing for efficient handling of high-dimensional vectors. By using Azure Cosmos DB\u2019s vector store, developers can keep vectors directly within their documents, enabling seamless integration with Spring AI or Langchain4J for advanced vector search queries.<\/p>\n

Spring AI vs Langchain4J<\/h3>\n

Langchain4J<\/a> and Spring AI<\/a> are both Java frameworks tailored for integrating AI capabilities into applications, but differ slightly in focus and approach. Both are designed to work seamlessly with large language models (LLMs), emphasizing modularity for constructing complex AI workflows like chaining prompts, handling memory, and managing vector stores. Spring AI<\/strong> is also a Spring ecosystem extension which brings along Spring Boot\u2019s<\/a> familiar configurations for ease of use. While Langchain4J excels in flexibility and advanced LLM-specific tools, Spring AI leverages Spring’s robust framework<\/a> ecosystem for quick, standardized integration, making it ideal for developers already invested in Spring-based projects, such as Azure Cosmos DB’s existing Spring Data Module<\/a>.<\/p>\n

Easy AI app development with Spring AI<\/h3>\n

The below examples are taken from the full sample here<\/a>. Creating an AI Chat Bot with Retrieval Augmented Generation (RAG)<\/a> using your own private data is easy in Spring AI once you have imported the required libraries:<\/p>\n

<dependencies>\r\n\t<dependency>\r\n\t\t<groupId>org.springframework.ai<\/groupId>\r\n\t\t<artifactId>spring-ai-core<\/artifactId>\r\n\t\t<version>1.0.0-M4<\/version>\r\n\t<\/dependency>\r\n\t<dependency>\r\n\t\t<groupId>org.springframework.ai<\/groupId>\r\n\t\t<artifactId>spring-ai-azure-cosmos-db-store-spring-boot-starter<\/artifactId>\r\n\t\t<version>1.0.0-M4<\/version>\r\n\t<\/dependency>\r\n\t<dependency>\r\n\t\t<groupId>org.springframework.ai<\/groupId>\r\n\t\t<artifactId>spring-ai-spring-boot-autoconfigure<\/artifactId>\r\n\t\t<version>1.0.0-M4<\/version>\r\n\t\t<scope>compile<\/scope>\r\n\t<\/dependency>\r\n\t<dependency>\r\n\t\t<groupId>org.springframework.ai<\/groupId>\r\n\t\t<artifactId>spring-ai-azure-openai<\/artifactId>\r\n\t\t<version>1.0.0-M4<\/version>\r\n\t\t<scope>compile<\/scope>\r\n\t<\/dependency>\r\n<\/dependencies><\/code><\/pre>\n
You can then create a configuration class for Spring AI’s ChatClient:<\/p>\n
package com.microsoft.azure.spring.chatgpt.sample.webapi;\r\n\r\nimport org.springframework.context.annotation.Bean;\r\nimport org.springframework.context.annotation.Configuration;\r\nimport org.springframework.ai.chat.client.ChatClient;\r\n\r\n@Configuration\r\npublic class ChatClientConfiguration {\r\n\r\n    private final ChatClient.Builder chatClientBuilder;\r\n\r\n    public ChatClientConfiguration(ChatClient.Builder chatClientBuilder) {\r\n        this.chatClientBuilder = chatClientBuilder;\r\n    }\r\n\r\n    @Bean\r\n    public ChatClient chatClient() {\r\n        return chatClientBuilder.build();\r\n    }\r\n}<\/code><\/pre>\n
Next, create a custom message class that implements Spring AI’s Message interface:<\/p>\n
package com.microsoft.azure.spring.chatgpt.sample.webapi.models;\r\n\r\nimport org.springframework.ai.chat.messages.Message;\r\nimport org.springframework.ai.chat.messages.MessageType;\r\n\r\nimport java.util.Map;\r\n\r\npublic class ChatMessage implements Message {\r\n    private String role;\r\n    private String content;\r\n\r\n    public ChatMessage(String role, String content) {\r\n        this.role = role;\r\n        this.content = content;\r\n    }\r\n\r\n    @Override\r\n    public String getContent() {\r\n        return content;\r\n    }\r\n\r\n    @Override\r\n    public Map<String, Object> getMetadata() {\r\n        return Map.of();\r\n    }\r\n\r\n    @Override\r\n    public MessageType getMessageType() {\r\n        return MessageType.USER;\r\n    }\r\n}<\/code><\/pre>\n
Then, add appropriate configuration in your application.properties file that will be autowired, like below:<\/p>\n
spring.ai.vectorstore.cosmosdb.databaseName=cosmosdb\r\nspring.ai.vectorstore.cosmosdb.containerName=vectorstore\r\nspring.ai.vectorstore.cosmosdb.partitionKeyPath=\/id\r\nspring.ai.vectorstore.cosmosdb.metadataFields=metadata\r\nspring.ai.vectorstore.cosmosdb.vectorStoreThoughput=1000\r\nspring.ai.vectorstore.cosmosdb.endpoint=${COSMOSDB_AI_ENDPOINT}\r\nspring.ai.vectorstore.cosmosdb.key=${COSMOSDB_AI_KEY}\r\nspring.ai.azure.openai.api-key=${AZURE_OPENAI_APIKEY}\r\nspring.ai.azure.openai.endpoint=${AZURE_OPENAI_ENDPOINT}\r\nspring.ai.azure.openai.embedding.options.deployment-name=text-embedding-ada-002<\/code><\/pre>\n
Finally, you can autowire VectorStore and ChatClient, and then do chat completion with RAG in one simple method within your controller. Chat completion is handled by your chatClient, and RAG is handled by calling the QuestionAnswerAdvisor on the chatClient. Here, we also add a custom method to transform the incoming messages into the expected ChatMessage format we defined above. Find out more about the powerful Spring AI Chat Client API<\/a> here.<\/p>\n
package com.microsoft.azure.spring.chatgpt.sample.webapi.controllers;\r\n\r\nimport com.fasterxml.jackson.core.JsonProcessingException;\r\nimport com.fasterxml.jackson.databind.ObjectMapper;\r\nimport com.microsoft.azure.spring.chatgpt.sample.webapi.models.ChatMessage;\r\nimport io.micrometer.observation.ObservationRegistry;\r\nimport org.springframework.ai.chat.client.ChatClient;\r\nimport org.springframework.ai.chat.client.advisor.QuestionAnswerAdvisor;\r\nimport org.springframework.ai.chat.messages.Message;\r\nimport org.springframework.ai.vectorstore.SearchRequest;\r\nimport org.springframework.ai.vectorstore.VectorStore;\r\nimport org.springframework.beans.factory.annotation.Autowired;\r\nimport org.springframework.context.annotation.Bean;\r\nimport org.springframework.context.annotation.Lazy;\r\nimport org.springframework.web.bind.annotation.PostMapping;\r\nimport org.springframework.web.bind.annotation.RequestBody;\r\nimport org.springframework.web.bind.annotation.RequestMapping;\r\nimport org.springframework.web.bind.annotation.RestController;\r\nimport java.util.List;\r\nimport java.util.stream.Collectors;\r\nimport java.util.stream.StreamSupport;\r\n\r\n@RestController\r\n@RequestMapping(\"\/chat\")\r\npublic class ChatController {\r\n\r\n    private static final String prompt = \"\"\"\r\n            Context information is below.\r\n            ---------------------\r\n            %s\r\n            ---------------------\r\n            Given the context information and not prior knowledge, answer the question: %s\r\n            \"\"\";\r\n\r\n    @Autowired\r\n    @Lazy\r\n    private VectorStore vectorStore;\r\n\r\n    @Autowired\r\n    @Lazy\r\n    private ChatClient chatClient;\r\n\r\n    @PostMapping(\"\/completions\")\r\n    public String generation(@RequestBody String userInput) throws JsonProcessingException {\r\n        List<Message> messages = getMessages(userInput);\r\n        String response = this.chatClient.prompt(prompt)\r\n                .user(messages.get(0).getContent())\r\n                .messages(messages)\r\n                .advisors(new QuestionAnswerAdvisor(vectorStore, SearchRequest.query(messages.get(0).getContent()).withTopK(5)))\r\n                .call()\r\n                .content();\r\n        return response;\r\n    }\r\n\r\n    @Bean\r\n    public ObservationRegistry observationRegistry() {\r\n        return ObservationRegistry.create();\r\n    }\r\n\r\n    public List<Message> getMessages(String text) throws JsonProcessingException {\r\n        ObjectMapper objectMapper = new ObjectMapper();\r\n        return StreamSupport.stream(objectMapper.readTree(text).get(\"messages\").spliterator(), false)\r\n                .map(node -> new ChatMessage(node.get(\"role\").asText(), node.get(\"content\").asText()))\r\n                .collect(Collectors.toList());\r\n    }\r\n}<\/code><\/pre>\n
Spring AI Integration – features and benefits<\/h3>\n
The Spring AI integration for Azure Cosmos DB brings some powerful benefits. Here are just a few:<\/p>\n
Efficient batch loading of vector embeddings<\/h4>\n
\n
When working with vector stores, it\u2019s often necessary to embed large numbers of documents. While it might seem straightforward to make a single call to embed all documents at once, this approach can lead to issues. Embedding models process text as tokens and have a maximum token limit, often referred to as the context window size. This limit restricts the amount of text that can be processed in a single embedding request. Attempting to embed too many tokens in one call can result in errors or truncated embeddings. To address this token limit, Spring AI implements a batching strategy. This approach breaks down large sets of documents into smaller batches that fit within the embedding model\u2019s maximum context window. Batching not only solves the token limit issue but can also lead to improved performance and more efficient use of API rate limits. In Azure Cosmos DB’s vector store implementation for Spring AI<\/a>, both bulk execution in the Java SDK and batching strategy in Spring AI are implemented, leading to efficient loading for large volumes of vectors. The above mentioned sample<\/a> includes a CLI application<\/a> for loading of vectors.<\/p>\n
Vector indexing with DiskANN<\/h4>\n
\n
\n
In Spring AI for Azure Cosmos DB, vector searches will create and leverage DiskANN indexes to ensure optimal performance for similarity queries. DiskANN (Disk-based Approximate Nearest Neighbor Search) is an innovative technology used in Azure Cosmos DB to enhance the performance of vector searches. It enables efficient and scalable similarity searches across high-dimensional data by indexing embeddings stored in Cosmos DB.<\/p>\n<\/div>\n
\n
DiskANN provides the following benefits:<\/p>\n<\/div>\n
\n