{"id":322,"date":"2023-04-25T08:14:14","date_gmt":"2023-04-25T15:14:14","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/semantic-kernel\/?p=322"},"modified":"2023-07-12T16:24:59","modified_gmt":"2023-07-12T23:24:59","slug":"the-power-of-persistent-memory-with-semantic-kernel-and-qdrant-vector-database","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/agent-framework\/the-power-of-persistent-memory-with-semantic-kernel-and-qdrant-vector-database\/","title":{"rendered":"The Power of Persistent Memory with Semantic Kernel and Qdrant Vector Database"},"content":{"rendered":"

\"Image<\/a><\/p>\n

A key component of leveraging machine learning large languages models (LLM) for natural language processing (NLP) in applications for chat search like ChatGPT, ranking and recommendation engines, anomaly detection, and semantic search is the ability to leverage massive amounts of unstructured data for search.<\/p>\n

But where would we save and retrieve all the unstructured data that powers these applications?<\/h2>\n

This is where vector databases enters the AI room. In order to efficiently and effectively, search, index, and store this unstructured data stored as embeddings, vector databases have risen to the challenge.<\/p>\n

Combining a vector database, the cloud, and a framework like Semantic Kernel becomes a powerful combination in building Generative AI applications. So, let\u2019s take a journey on how to use the semantic search and vector database, Qdrant<\/a>, running on Azure cloud and integrated with Semantic Kernel to enable a generative AI solution.<\/p>\n

Like any journey, preparation is key. Therefore, before starting we need to review our checklist to ensure that all the bases are covered:<\/p>\n

    \n
  1. Is there a valid Azure Subscription that can be used? No worries if there is not an Azure subscription at your disposable, a free Azure subscription can be created by going to Create Your Azure Free Account Today | Microsoft Azure<\/a><\/li>\n
  2. Am I successfully logged into the Azure Subscription that will be used?<\/li>\n
  3. Do I have permissions and scope set up in the Azure Subscription I have access to, to create a resource group and create resources like AKS? This is a big one, in working with customers we\u2019ve found that the not having the correct permissions and scopes has been the biggest challenge when deploying solutions. In starting off this journey in the correct way, it\u2019s essential to check if your account has the correct access.<\/li>\n<\/ol>\n

    \"Qdrant<\/a><\/p>\n

    Once you have navigated to the GitHub repo, the repo should be cloned using the method you are most comfortable with. I\u2019m a command line type of person, so I do this by executing the following command in bash shell:<\/p>\n

    % git clone https:\/\/github.com\/Azure-Samples\/qdrant-azure.git<\/code><\/p>\n

    Now that the repo is cloned, let\u2019s create a Azure resource group to put the resources to be created to run Qdrant in the Azure cloud. I do with the Azure CLI in bash by issuing command:<\/p>\n

    % az group create --name az-qdrant-sk-rg --location eastus<\/code><\/p>\n

    It will return a JSON snippet if successful:<\/p>\n

    {\n\"id\": \"\/subscriptions\/...\n   ...\n   \"type\": ...\n}<\/code><\/p>\n

    Alternatively, an Azure resource group could be easily created in the Azure Portal as well.<\/p>\n

    \"Image<\/a><\/p>\n

    Let\u2019s also create an Azure SSHKey that we will need later for deployment. Create an Azure SSH key in the Resource group created in the previous step by executing the following command<\/p>\n

    % <\/strong>az sshkey create --name \"qdrantSSHKey\" --resource-group \"<INSERT YOUR ResourceGroup<\/em>>\"<\/code><\/p>\n

    via the command line or via Azure Cloud Shell. Again, a JSON snippet will be returned of the form:<\/p>\n

    {\n   \"id\": \"\/subscriptions\/...\n   ...\n   \"type\": null\n}<\/code><\/p>\n

    Congratulations! You’ve created an SSHKey in Azure within your Resource Group.<\/p>\n

    Looks like we\u2019re off to the races …<\/h2>\n

    Next stop, we will got back the Qdrant on Azure GitHub repo<\/a> we pulled up in the browser. On the README in the repo, is an ARM (Azure Resource Manager) button.<\/p>\n

    Click the ARM button, Deploy to Azure<\/strong>, to create an Kubernetes cluster in Azure to run Qdrant. Note<\/strong>: If you already have a AKS cluster set up or a K8s cluster running locally you can skip this step. <\/em><\/strong><\/p>\n

    A Custom Deployment screen will open up in Azure Portal. Use the dropdown for Resource group<\/strong> to select the Azure Resource group we created previously. Add a value\/name for DNS Prefix<\/strong>, and lastly under the SSH public key source option select the Use an existing key stored in Azure <\/strong>option. Select the Azure SSHKey that was created in the earlier in our journey.\"Image<\/a><\/p>\n

     <\/p>\n

    A Custom Deployment screen will open up in Azure Portal. Use the dropdown for Resource group<\/strong> to select the Azure Resource group we created previously. Add a value\/name for DNS Prefix<\/strong>, and lastly under the SSH public key source option select the Use an existing key stored in Azure <\/strong>option.\u00a0Select the Azure SSHKey that was created in the earlier in our journey in the Stored Keys<\/strong> dropdown.<\/p>\n

    \"Image<\/a><\/p>\n

    Select Review + Create<\/strong> button, to Validate your deployment and all that is left here is to select Create<\/strong> button deploy your AKS cluster for Qdrant.<\/p>\n

    \"Image<\/a><\/p>\n

    \"Image<\/a><\/p>\n

    We\u2019re almost at the end of our journey, only 2 steps remain …<\/h2>\n

    First is to ensure Qdrant<\/strong> is deployed to our AKS cluster by running the Helm chart located in the Qdrant Azure Github repo<\/strong> we cloned earlier and the second is to tie Qdrant<\/strong> vector database to the Semantic Kernel.<\/p>\n

    To accomplish the first step, go to the location you cloned the repo. From the root directory of the repo, change to the directory Azure-Kubernetes-Svc<\/strong>. We\u2019re going to run commands to get the AKS credentials stored locally and to execute the Helm chart. Don\u2019t worry, if those words are foreign to you; there are scripts that will do all the heavy lifting for you.<\/p>\n

    To get the tools, Kubectl<\/strong> and Helm<\/strong>, needed to get your Qdrant<\/strong> instance running correctly, follow the instructions provided in repo here: https:\/\/github.com\/Azure-Samples\/qdrant-azure\/tree\/main\/Azure-Kubernetes-Svc#prerequisites<\/a><\/p>\n

    Now, that we\u2019ve got the tools and ready to rock and roll, let’s start deploying Qdrant to our AKS cluster. Issue the following command on the command line using Azure CLI:<\/p>\n

    % az aks get-credentials --resource-group <your-resource-group-name><\/strong><\/em> --name azqdrantcluster<\/code>\n<\/code><\/strong>**if you used the template, azqdrantcluster will be the name of your AKS cluster**<\/em><\/span><\/p>\n

    Then execute the following Kubectl<\/strong> command on the command line: <\/code><\/p>\n

    % kubectl get nodes<\/code><\/p>\n

    This will verify that you can access your cluster and it\u2019s running successfully.<\/p>\n

    Finally, let\u2019s get Qdrant humming on your cluster. Navigate to the qdrant-on-azure<\/strong> folder and run the following command. <\/code><\/p>\n

    % helm install azqdrant-sk ..\/qdrant-on-azure\/ --create-namespace<\/code><\/p>\n

    or for a custom namespace you can run<\/code><\/p>\n

    % helm install azqdrant-sk ..\/qdrant-on-azure --namespace test-namespace --create-namespace<\/code><\/p>\n

    And voil\u00e0 we have Qdrant<\/strong> running on Azure<\/strong>.<\/p>\n

    Let’s get the IP address that will be used for Qdrant API calls by running: kubectl get services<\/code><\/strong>. Make sure to copy the External-IP<\/strong> of the azure-qdrant-loadbalancer<\/strong> as this will be needed in our final step in using Qdrant<\/strong> with the Semantic Kernel. See screenshots below for an example.<\/p>\n

    default namespace<\/em><\/strong><\/p>\n

    \"Image<\/a><\/p>\n

    with custom namespace<\/em><\/strong><\/p>\n

    \"Image<\/a><\/p>\n

    I was always taught that it’s better to be safe than sorry in both engineering and in life. Therefore, let’s make sure that the Qdrant instance is functional and running as desired by creating a test collection. To test that Qdrant<\/strong> is running successfully, run the following command on the command line:<\/p>\n

    % curl -X PUT <\/code>'http:\/\/<your QDRANT-LOADBALANCER-EXTERNALIP-ADDRESS><\/em><\/strong>:6333\/collections\/test_collection'<\/span> \\ -H 'Content-Type: application\/json'<\/span> \\ --data-raw '{ \"vectors\": { \"size\": 4, \"distance\": \"Dot\" } }'<\/span><\/code><\/p>\n

    If this executes correctly you should see a json payload that shows something similar to:\u00a0 {\"result\":true,\"status\":\"ok\",\"time\":0.407534142} <\/code><\/p>\n

    See screenshot below for an example of the successful creation of your test Qdrant<\/strong> collection.<\/p>\n

    <\/code>testing of Qdrant creation of collection<\/em><\/strong><\/p>\n

    \"Image<\/a><\/p>\n

    Wow, we\u2019re at the final leg of our journey and we\u2019ve found our way to the Semantic Kernel with our Qdrant<\/strong> IP address in hand …<\/h2>\n

    Go to the GitHub repo for the Semantic Kernel<\/strong> located here: https:\/\/github.com\/microsoft\/semantic-kernel<\/a>. Clone the repo using your method of choice for Git. Follow the steps noted in the readme to setup the semantic kernel.<\/p>\n

    Now that we\u2019re all setup, we can run the sample for using Qdrant<\/strong> Memory with the Semantic Kernel. Go the folder named kernel-syntax-examples<\/strong> by navigating to: semantic-kernel\/samples\/dotnet\/kernel-syntax-examples\/ <\/strong>from the location\/directory you cloned the Semantic Kernel repo. Under that folder should see a C# project files and a list of code files with the heading Examples.<\/p>\n

    Open up the KernelSyntaxExamples.csproj<\/strong> project in an IDE like Visual Studio Code<\/a> and look for Example19_Qdrant.cs<\/strong>. This is the sample that will be run with the Qdrant <\/strong>instance we created.<\/p>\n

    To set up the environment variables needed to run the sample, grab your OpenAI API key<\/strong> from your OpenAI<\/strong> account from https:\/\/platform.openai.com\/account\/api-keys<\/a> or from your Azure subscription’s Azure OpenAI Service.<\/a> Also have in hand, the Qdrant External IP <\/strong>address we obtained earlier from Qdrant<\/strong> instance. Run the following commands on the command line:<\/p>\n

    % dotnet user-secrets set \"OPENAI_API_KEY\" \"<Insert your OpenAI API Key<\/em>><\/strong>\u201d<\/code><\/p>\n

    % dotnet user-secrets set \"QDRANT_ENDPOINT\" \"http:\/\/<Insert your Qdrant External-IP address<\/em><\/strong>>\"<\/code><\/p>\n

    % dotnet user-secrets set \"QDRANT_PORT\" \"6333\"<\/code><\/p>\n

    In your open project, go to the Program.cs<\/strong> file to the Program class, you will see a list of the examples available. Comment out (or remove) all the samples except for the command for the Example19_Qdrant.cs<\/strong> sample which is the code line await Example19_Qdrant.RunAsync();<\/code> in the Program<\/strong> class. <\/span><\/p>\n

    See screenshots as a visual of what this will look like in Visual Studio Code.<\/span><\/p>\n

    \n\"Image<\/a>\n<\/span><\/p>\n

    Build\/Compile the code and run the sample. You have now successfully implemented Qdrant<\/strong> with the Semantic Kernel<\/strong>.<\/p>\n

    Thanks for taking the ride with me.<\/h2>\n\n\n\n
    \"Image<\/a><\/td>\nTara E. Walker<\/strong>, Principal Software Engineer @Microsoft<\/p>\n

    Tara is a Principal Software Engineer in the Office of the CTO (OCTO) developing solutions using the latest AI\/ML workloads and cutting edge technologies at scale including Cloud Computing\/Cloud Native and Robotics. Prior to Microsoft she was at AWS and Turner Broadcasting\/Time Warner.<\/em><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

     <\/p>\n

    \"Image<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

    A key component of leveraging machine learning large languages models (LLM) for natural language processing (NLP) in applications for chat search like ChatGPT, ranking and recommendation engines, anomaly detection, and semantic search is the ability to leverage massive amounts of unstructured data for search. But where would we save and retrieve all the unstructured data […]<\/p>\n","protected":false},"author":117489,"featured_media":370,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[17,1,19],"tags":[],"class_list":["post-322","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-announcements","category-semantic-kernel","category-vector-database"],"acf":[],"blog_post_summary":"

    A key component of leveraging machine learning large languages models (LLM) for natural language processing (NLP) in applications for chat search like ChatGPT, ranking and recommendation engines, anomaly detection, and semantic search is the ability to leverage massive amounts of unstructured data for search. But where would we save and retrieve all the unstructured data […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/322","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/users\/117489"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/comments?post=322"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/322\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media\/370"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media?parent=322"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/categories?post=322"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/tags?post=322"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}