{"id":4503,"date":"2025-03-14T10:32:20","date_gmt":"2025-03-14T17:32:20","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/semantic-kernel\/?p=4503"},"modified":"2025-03-14T10:32:20","modified_gmt":"2025-03-14T17:32:20","slug":"customer-case-study-announcing-the-microsoft-semantic-kernel-couchbase-connector","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/agent-framework\/customer-case-study-announcing-the-microsoft-semantic-kernel-couchbase-connector\/","title":{"rendered":"Customer Case Study: Announcing the Microsoft Semantic Kernel Couchbase Connector"},"content":{"rendered":"<p>We&#8217;re thrilled to announce the launch of the <a href=\"https:\/\/github.com\/Couchbase-Ecosystem\/couchbase-semantic-kernel\">Semantic Kernel Couchbase Vector Store Connector<\/a> for .NET developers, created through our strategic partnership with Microsoft&#8217;s Semantic Kernel team. This powerful <a href=\"https:\/\/learn.microsoft.com\/en-us\/semantic-kernel\/concepts\/vector-store-connectors\/out-of-the-box-connectors\/couchbase-connector?pivots=programming-language-csharp\">out-of-the-box<\/a> connector transforms how developers integrate vector search capabilities into their AI applications.<\/p>\n<p>What sets this connector apart is how it harnesses Couchbase&#8217;s distributed NoSQL platform alongside Semantic Kernel&#8217;s <a href=\"https:\/\/learn.microsoft.com\/en-us\/semantic-kernel\/concepts\/vector-store-connectors\/?pivots=programming-language-csharp#the-vector-store-abstraction\">vector store abstractions<\/a>, creating an integration that prioritizes both performance and developer experience.<\/p>\n<p>The Semantic Kernel Couchbase Vector Store Connector eliminates traditional barriers between data storage and AI processing, giving developers the freedom to focus on creating intelligent, context-aware applications.<\/p>\n<p>In this blog, we&#8217;ll explore how this connector enhances AI development workflows, demonstrating practical examples of how you can leverage Couchbase&#8217;s vector capabilities to build more responsive, data-driven AI agents.<\/p>\n<h2>Microsoft Semantic Kernel and Couchbase<\/h2>\n<p><a href=\"https:\/\/learn.microsoft.com\/en-us\/semantic-kernel\/overview\/\">Semantic Kernel<\/a> is a lightweight, open-source development kit that lets you easily build AI agents and integrate AI models into your codebase. It serves as efficient middleware that enables rapid delivery of enterprise-grade solutions. By combining prompts with existing APIs, Semantic Kernel allows AI models to perform actions through your code &#8211; when a request is made, the model calls a function, and Semantic Kernel translates this request into a function call and returns the results to the model. Its modular design lets you add your existing code as plugins, maximizing your investment through flexible integration.<\/p>\n<p>The new Couchbase connector extends these capabilities by seamlessly integrating Couchbase\u2014whether via <a href=\"https:\/\/docs.couchbase.com\/server\/current\/getting-started\/start-here.html\">Couchbase Server<\/a> or <a href=\"https:\/\/docs.couchbase.com\/cloud\/get-started\/create-account.html\">Couchbase Capella<\/a>\u2014into the Semantic Kernel environment. It delivers efficient data storage, retrieval, and similarity search for high-dimensional embeddings using robust indexing and flexible JSON document management, enabling developers to quickly build responsive, enterprise-grade AI applications with minimal code changes.<\/p>\n<h2>Prerequisites<\/h2>\n<h3>Create and Deploy Your Free Tier Operational cluster on Capella<\/h3>\n<p>To get started with Couchbase Capella, create an account and use it to deploy a forever free tier operational cluster. This account provides you with an environment where you can explore and learn about Capella with no time constraint.<\/p>\n<p>To know more, please follow the <a href=\"https:\/\/docs.couchbase.com\/cloud\/get-started\/create-account.html\">instructions<\/a>.<\/p>\n<h3>Couchbase Capella Configuration<\/h3>\n<p>When running Couchbase using <a href=\"https:\/\/cloud.couchbase.com\/sign-in\">Capella<\/a>, the following prerequisites need to be met.<\/p>\n<ul>\n<li>Create the <a href=\"https:\/\/docs.couchbase.com\/cloud\/clusters\/manage-database-users.html\">database credentials<\/a> to access the travel-sample bucket (Read and Write) used in the application.<\/li>\n<li><a href=\"https:\/\/docs.couchbase.com\/cloud\/clusters\/allow-ip-address.html\">Allow access<\/a> to the Cluster from the IP on which the application is running.<\/li>\n<\/ul>\n<h3>Creating Vector Search Index<\/h3>\n<p>For creating a vector search index, please follow the <a href=\"https:\/\/docs.couchbase.com\/cloud\/vector-search\/create-vector-search-index-ui.html\">instructions<\/a>. Here is a <a href=\"https:\/\/github.com\/Couchbase-Ecosystem\/couchbase-semantic-kernel\/blob\/main\/Couchbase.SemanticKernel.Playground\/index.json\">sample index<\/a> that you can create considering the below RAG example.<\/p>\n<p>In this sample, the index is configured on the <strong>travel-sample<\/strong> bucket within the <strong>inventory<\/strong> scope, where a collection named <strong>semantickernel<\/strong> is created. This collection is used to store the data for the RAG example. The index configuration allows you to set the similarity function\u2014such as dot product\u2014and specify the dimensions of the embedding field (e.g., 1536). This flexibility lets you tailor the index to your application\u2019s specific requirements.<\/p>\n<p>If you wish to map the <a href=\"https:\/\/learn.microsoft.com\/en-us\/semantic-kernel\/concepts\/vector-store-connectors\/defining-your-data-model?pivots=programming-language-csharp#vectorstorerecorddataattribute-parameters\">IsFilterable<\/a><a href=\"https:\/\/learn.microsoft.com\/en-us\/semantic-kernel\/concepts\/vector-store-connectors\/defining-your-data-model?pivots=programming-language-csharp#vectorstorerecorddataattribute-parameters\"> and <\/a><a href=\"https:\/\/learn.microsoft.com\/en-us\/semantic-kernel\/concepts\/vector-store-connectors\/defining-your-data-model?pivots=programming-language-csharp#vectorstorerecorddataattribute-parameters\">IsFullTextSearchable<\/a><a href=\"https:\/\/learn.microsoft.com\/en-us\/semantic-kernel\/concepts\/vector-store-connectors\/defining-your-data-model?pivots=programming-language-csharp#vectorstorerecorddataattribute-parameters\"> attributes<\/a>, you can configure them as shown below. In this example, the <span style=\"font-family: 'courier new', courier, monospace;\">HotelName<\/span> field is set to be filterable by using the <span style=\"font-family: 'courier new', courier, monospace;\">keyword<\/span> analyzer for exact matching, while the <span style=\"font-family: 'courier new', courier, monospace;\">Description<\/span> field is configured for full-text search using the <span style=\"font-family: 'courier new', courier, monospace;\">standard<\/span> analyzer. For more information on available analyzers, please refer to the <a href=\"https:\/\/docs.couchbase.com\/server\/current\/search\/default-analyzers-reference.html\">documentation<\/a>.<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">{\r\n  \"types\": {\r\n    \"inventory.semantickernel\": {\r\n      \"dynamic\": false,\r\n      \"enabled\": true,\r\n      \"properties\": {\r\n        \"description\": {\r\n          \"dynamic\": false,\r\n          \"enabled\": true,\r\n          \"fields\": [\r\n            {\r\n             \"analyzer\": \"standard\",  \/\/ using standard analyzer for full-text search\r\n              \"index\": true,\r\n              \"name\": \"description\",\r\n              \"store\": true,\r\n              \"type\": \"text\"\r\n            }\r\n          ]\r\n        },\r\n        \"hotelName\": {\r\n          \"dynamic\": false,\r\n          \"enabled\": true,\r\n          \"fields\": [\r\n            {\r\n            \"analyzer\": \"keyword\",  \/\/ using keyword analyzer for exact matching\r\n              \"index\": true,\r\n              \"name\": \"hotelName\",\r\n              \"store\": true,\r\n              \"type\": \"text\"\r\n            }\r\n          ]\r\n        }\r\n        \/\/ ... other fields ...\r\n      }\r\n    }\r\n  }\r\n}\r\n<\/code><\/pre>\n<h2>High-Level Scenario &amp; RAG Example<\/h2>\n<p>In this demonstration, we build a Retrieval Augmented Generation (RAG) application:<\/p>\n<ul>\n<li><strong>User Input:<\/strong> A user submits a question.<\/li>\n<li><strong>Processing:<\/strong> The application generates an embedding for the question and retrieves relevant entries from the Couchbase vector store.<\/li>\n<li><strong>Output:<\/strong> The LLM uses the retrieved context to generate a detailed answer, including data source references.<\/li>\n<\/ul>\n<p>For instance, imagine an application where users can query a hotel database. In our demo, we generated <a href=\"https:\/\/github.com\/Couchbase-Ecosystem\/couchbase-semantic-kernel\/blob\/main\/Couchbase.SemanticKernel.Playground\/hotels.csv\">100 sample hotel entries<\/a>\u2014an intentionally small dataset to let you easily try out the connector demo. In a production scenario, however, the Couchbase connector would clearly demonstrate its superior performance and scalability, especially when handling very large datasets.<\/p>\n<p>The complete demo application is available in the Couchbase vector store connector <a href=\"https:\/\/github.com\/Couchbase-Ecosystem\/couchbase-semantic-kernel\/tree\/main\/Couchbase.SemanticKernel.Playground\">repository<\/a>.<\/p>\n<h3>Project Setup<\/h3>\n<p>Add the required NuGet packages:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">dotnet add package \"CouchbaseConnector.SemanticKernel\" -v 0.2.2\r\ndotnet add package \"Microsoft.Extensions.Hosting\" -v 9.0.2\r\ndotnet add package \"Microsoft.SemanticKernel.PromptTemplates.Handlebars\" -v 1.40.0\r\n<\/code><\/pre>\n<p>Include the necessary using directives:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">using System.Text.Json.Serialization;\r\nusing Microsoft.Extensions.DependencyInjection;\r\nusing Microsoft.Extensions.Hosting;\r\nusing Microsoft.Extensions.VectorData;\r\nusing Microsoft.SemanticKernel;\r\nusing Microsoft.SemanticKernel.Data;\r\nusing Microsoft.SemanticKernel.Embeddings;\r\nusing Microsoft.SemanticKernel.PromptTemplates.Handlebars;\r\n<\/code><\/pre>\n<h3>Defining the Data Model<\/h3>\n<p>Create your data model for a hotel record:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">\/\/\/ &lt;summary&gt;\r\n\/\/\/ Data model for storing a \"hotel\" with a name, a description, a description embedding and an optional reference link.\r\n\/\/\/ &lt;\/summary&gt;\r\npublic sealed record Hotel\r\n{\r\n    [VectorStoreRecordKey]\r\n    [JsonPropertyName(\"hotelId\")]\r\n    public required string HotelId { get; set; }\r\n\r\n    [TextSearchResultName]\r\n    [VectorStoreRecordData]\r\n    [JsonPropertyName(\"hotelName\")]\r\n    public required string HotelName { get; set; }\r\n\r\n    [TextSearchResultValue]\r\n    [VectorStoreRecordData]\r\n    [JsonPropertyName(\"description\")]\r\n    public required string Description { get; set; }\r\n\r\n[VectorStoreRecordVector(Dimensions: 1536, DistanceFunction.DotProductSimilarity)]\r\n    [JsonPropertyName(\"descriptionEmbedding\")]\r\n    public ReadOnlyMemory&lt;float&gt; DescriptionEmbedding { get; set; }\r\n\r\n    [TextSearchResultLink]\r\n    [VectorStoreRecordData]\r\n    [JsonPropertyName(\"referenceLink\")]\r\n    public string? ReferenceLink { get; set; }\r\n}\r\n<\/code><\/pre>\n<p>The <span style=\"font-family: 'courier new', courier, monospace;\">VectorStore<\/span>* attributes define the storage schema, while the <span style=\"font-family: 'courier new', courier, monospace;\">TextSearch<\/span>* attributes enable dynamic text search in prompt templates.<\/p>\n<h3>Initializing the Semantic Kernel Engine<\/h3>\n<p>Set up the engine and register Couchbase settings (loaded from <span style=\"font-family: 'courier new', courier, monospace;\">appsettings.json<\/span>):<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">var builder = Host.CreateApplicationBuilder(args);\r\n\/\/ Add configuration from appsettings.json\r\nvar couchbaseConfig = builder.Configuration.GetSection(\"Couchbase\");\r\n\/\/ Register AI services.\r\nvar kernelBuilder = builder.Services.AddKernel();\r\nkernelBuilder.AddAzureOpenAIChatCompletion(\"gpt-4o\", \"https:\/\/my-service.openai.azure.com\", \"my_token\");\r\n        kernelBuilder.AddAzureOpenAITextEmbeddingGeneration(\"ada-002\", \"https:\/\/my-service.openai.azure.com\", \"my_token\");\r\n\/\/ Register text search service.\r\nkernelBuilder.AddVectorStoreTextSearch&lt;Hotel&gt;();\r\n\/\/ Register Couchbase Vector Store using provided extensions.\r\nbuilder.Services.AddCouchbaseFtsVectorStoreRecordCollection&lt;Hotel&gt;(\r\n    connectionString: couchbaseConfig[\"ConnectionString\"],\r\n    username: couchbaseConfig[\"Username\"],\r\n    password: couchbaseConfig[\"Password\"],\r\n    bucketName: couchbaseConfig[\"BucketName\"],\r\n    scopeName: couchbaseConfig[\"ScopeName\"],\r\n    collectionName: couchbaseConfig[\"CollectionName\"],\r\n    options: new CouchbaseFtsVectorStoreRecordCollectionOptions&lt;Hotel&gt;\r\n    {\r\n        IndexName = couchbaseConfig[\"IndexName\"]\r\n    });\r\n\/\/ Build the host.\r\nusing var host = builder.Build();\r\n\/\/ Access services directly (for demo purposes).\r\nvar kernel = host.Services.GetService&lt;Kernel&gt;()!;\r\nvar embeddings = host.Services.GetService&lt;ITextEmbeddingGenerationService&gt;()!;\r\nvar vectorStoreCollection = host.Services.GetService&lt;IVectorStoreRecordCollection&lt;string, Hotel&gt;&gt;()!;\r\n\/\/ Register search plugin.\r\nvar textSearch = host.Services.GetService&lt;VectorStoreTextSearch&lt;Hotel&gt;&gt;()!;\r\nkernel.Plugins.Add(textSearch.CreateWithGetTextSearchResults(\"SearchPlugin\"));\r\n<\/code><\/pre>\n<h3>Ingesting Demo Data<\/h3>\n<p>Load demo hotel records (CSV format: ID;Hotel Name;Description;Reference Link) and ingest them into the vector store:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">\/\/ CSV format: ID;Hotel Name;Description;Reference Link\r\nvar hotels = (await File.ReadAllLinesAsync(filePath))\r\n             .Select(x =&gt; x.Split(';'));\r\n\r\nforeach (var chunk in hotels.Chunk(25))\r\n{\r\n    var descriptionEmbeddings = await embeddings.GenerateEmbeddingsAsync(chunk.Select(x =&gt; x[2]).ToArray());\r\n             \r\n    for (var i = 0; i &lt; chunk.Length; ++i)\r\n    {\r\n        var hotel = chunk[i];\r\n        await vectorStoreCollection.UpsertAsync(new Hotel\r\n        {\r\n            HotelId = hotel[0],\r\n            HotelName = hotel[1],\r\n            Description = hotel[2],\r\n            DescriptionEmbedding = descriptionEmbeddings[i],\r\n            ReferenceLink = hotel[3]\r\n        });\r\n    }\r\n}\r\n<\/code><\/pre>\n<p>The <span style=\"font-family: 'courier new', courier, monospace;\">embeddings.GenerateEmbeddingsAsync()<\/span> method transparently calls the configured Azure AI Embeddings Generation service.<\/p>\n<h3>Invoking the AI Agent<\/h3>\n<p>Invoke the LLM with a prompt that integrates search results:<\/p>\n<pre class=\"prettyprint language-default\"><code class=\"language-default\">var response = await kernel.InvokePromptAsync(\r\n  promptTemplate: \"\"\"\r\n                  Please use this information to answer the question:\r\n                  {{#with (SearchPlugin-GetTextSearchResults question)}}\r\n                    {{#each this}}\r\n                      Name: {{Name}}\r\n                      Value: {{Value}}\r\n                      Source: {{Link}}\r\n                      -----------------\r\n                    {{\/each}}\r\n                  {{\/with}}\r\n\r\n                  Include the source of relevant information in the response.\r\n\r\n                  Question: {{question}}\r\n                  \"\"\",\r\n  arguments: new KernelArguments\r\n  {\r\n      { \"question\", \"Please show me all hotels that have a rooftop bar.\" },\r\n  },\r\n  templateFormat: \"handlebars\",\r\n  promptTemplateFactory: new HandlebarsPromptTemplateFactory());\r\nConsole.WriteLine(response.ToString());\r\n<\/code><\/pre>\n<p>The <span style=\"font-family: 'courier new', courier, monospace;\">TextSearch<\/span>* attributes we set in our data model\u2014they automatically populate our prompt templates with the relevant information from the vector store through the corresponding placeholders. When a user queries, for example, \u201cPlease show me all hotels that have a rooftop bar,\u201d the system processes the request by generating an embedding for the question, searching the vector store, and then constructing the answer based on our template. Running the code (e.g., via Console.WriteLine(response.ToString());) will output similar to this:<\/p>\n<p><span style=\"font-family: 'courier new', courier, monospace; font-size: 10pt;\">&#8211; **Skyline Suites**: Offering panoramic city views from every suite, this hotel includes a rooftop bar. [Source](https:\/\/example.com\/yz567)<\/span><\/p>\n<p>This result correctly corresponds to the hotel record in our CSV file.<\/p>\n<p><span style=\"font-family: 'courier new', courier, monospace; font-size: 10pt;\">9;<\/span><\/p>\n<p><span style=\"font-family: 'courier new', courier, monospace; font-size: 10pt;\">Skyline Suites;<\/span><\/p>\n<p><span style=\"font-family: 'courier new', courier, monospace; font-size: 10pt;\">Offering panoramic city views from every suite, this hotel is perfect for those who love the urban landscape. Enjoy luxurious amenities, a rooftop bar, and close proximity to attractions. Luxurious and contemporary.;<\/span><\/p>\n<p><span style=\"font-family: 'courier new', courier, monospace; font-size: 10pt;\"><a href=\"https:\/\/example.com\/yz567\">https:\/\/example.com\/yz567<\/a><\/span><\/p>\n<p>This demonstration highlights how Microsoft Semantic Kernel\u2019s well-designed abstractions simplify the integration process while offering great flexibility. The framework\u2019s robust features\u2014such as the InvokePrompt function, along with its templating and search plugin systems\u2014streamline the development of intelligent, responsive applications.<\/p>\n<h3>What\u2019s next?<\/h3>\n<ul>\n<li><strong>Hybrid Search with Semantic Kernel:<\/strong> We&#8217;re thrilled to partner with Microsoft to bring <a href=\"https:\/\/www.couchbase.com\/blog\/hybrid-search\/\">hybrid search<\/a> and advanced retrieval strategies to Semantic Kernel developers in the near future.<\/li>\n<\/ul>\n<p>The Microsoft Semantic Kernel Couchbase Connector is a transformative tool for AI agent development. By merging Couchbase\u2019s robust, scalable storage with the abstractions of Microsoft Semantic Kernel, developers can rapidly build intelligent, responsive applications with minimal overhead. Explore the complete demo in the Couchbase vector store connector repository and join us as we continue to innovate in the realm of AI-driven solutions.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We&#8217;re thrilled to announce the launch of the Semantic Kernel Couchbase Vector Store Connector for .NET developers, created through our strategic partnership with Microsoft&#8217;s Semantic Kernel team. This powerful out-of-the-box connector transforms how developers integrate vector search capabilities into their AI applications. What sets this connector apart is how it harnesses Couchbase&#8217;s distributed NoSQL platform [&hellip;]<\/p>\n","protected":false},"author":149071,"featured_media":4508,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[42],"tags":[48,129,130,63,9],"class_list":["post-4503","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-customer-story","tag-ai","tag-couchbase","tag-couchbase-connector","tag-microsoft-semantic-kernel","tag-semantic-kernel"],"acf":[],"blog_post_summary":"<p>We&#8217;re thrilled to announce the launch of the Semantic Kernel Couchbase Vector Store Connector for .NET developers, created through our strategic partnership with Microsoft&#8217;s Semantic Kernel team. This powerful out-of-the-box connector transforms how developers integrate vector search capabilities into their AI applications. What sets this connector apart is how it harnesses Couchbase&#8217;s distributed NoSQL platform [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/4503","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/users\/149071"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/comments?post=4503"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/4503\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media\/4508"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media?parent=4503"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/categories?post=4503"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/tags?post=4503"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}