{"id":45823,"date":"2023-05-23T09:45:00","date_gmt":"2023-05-23T16:45:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/dotnet\/?p=45823"},"modified":"2024-12-13T14:15:35","modified_gmt":"2024-12-13T22:15:35","slug":"transform-business-smart-dotnet-apps-azure-chatgpt","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/transform-business-smart-dotnet-apps-azure-chatgpt\/","title":{"rendered":"Transform your business with smart .NET apps powered by Azure and ChatGPT"},"content":{"rendered":"<p>With ChatGPT, you can unleash the full potential of AI in your .NET applications and create amazing user experiences with natural language. ChatGPT is more than just a tool; it\u2019s a game-changer for how we access and analyze data. Whether you use Azure, SQL Server, or any other data source, you can easily integrate ChatGPT into your .NET projects and start building intelligent apps today.<\/p>\n<p>In this post, I\u2019ll provide a quick overview of what intelligent applications are. Then, using a sample application, I\u2019ll show how using a combination of Azure services like <a href=\"https:\/\/learn.microsoft.com\/azure\/cognitive-services\/openai\/overview\">Azure OpenAI Service<\/a> and <a href=\"https:\/\/learn.microsoft.com\/azure\/search\/search-what-is-azure-search\">Azure Cognitive Search<\/a>, you can build your own intelligent .NET applications.<\/p>\n<h2>TLDR<\/h2>\n<p>Do you learn best from experience? Head over the repo and jump right in!<\/p>\n<p><div  class=\"d-flex justify-content-center\"><a class=\"cta_button_link btn-primary mb-24\" href=\"https:\/\/aka.ms\/dotnet-ai-app\" target=\"_blank\">Build Your Own .NET Intelligent App<\/a><\/div><\/p>\n<h2>What are intelligent applications?<\/h2>\n<p>Intelligent applications are AI-powered applications that transform users\u2019 productivity, automate processes, and derive insights.<\/p>\n<p>Bing Chat is an example of an intelligent application.<\/p>\n<p><img decoding=\"async\" src=\".\/bing-chat.png\" alt=\"Image of Bing Chat conversation about recipes\" \/><\/p>\n<p>AI is at the core of Bing Chat. Bing Chat uses AI to take in complex queries, summarize relevant information from various sources, and generates responses. <\/p>\n<h2>Build intelligent apps with .NET<\/h2>\n<p>Now that you have a sense of what intelligent applications are, let\u2019s look at a sample application built with .NET, Azure, and ChatGPT.<\/p>\n<p><div style=\"width: 640px;\" class=\"wp-video\"><video class=\"wp-video-shortcode\" id=\"video-45823-1\" width=\"640\" height=\"360\" preload=\"metadata\" controls=\"controls\"><source type=\"video\/mp4\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/05\/dotnet-openai-chat-app.mp4?_=1\" \/><a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/05\/dotnet-openai-chat-app.mp4\">https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/05\/dotnet-openai-chat-app.mp4<\/a><\/video><\/div><\/p>\n<p>Let\u2019s say you have an internal enterprise knowledgebase which contains information about job roles, health care plans, and other business documents.<\/p>\n<p>Your users may already be able to search through this knowledgebase, but searching for answers to specific questions by filtering through all the documents in the search results can be time-consuming. <\/p>\n<p>Using AI models like ChatGPT, you can transform your user\u2019s productivity by summarizing the information contained in those documents and extracting key insights.  <\/p>\n<h3>Application architecture<\/h3>\n<p>The <a href=\"https:\/\/aka.ms\/dotnet-ai-app\">source code<\/a> for the application is on GitHub. The following are the core components that make up the application. <\/p>\n<p><img decoding=\"async\" src=\".\/app-architecture.svg\" alt=\"Architectural diagram of ChatGPT .NET intelligent app\" \/><\/p>\n<h4>User interface<\/h4>\n<p>The application\u2019s chat interface is a <a href=\"https:\/\/learn.microsoft.com\/aspnet\/core\/blazor\/?WT.mc_id=dotnet-35129-website&amp;view=aspnetcore-7.0#blazor-webassembly\">Blazor WebAssembly<\/a> static web application. This interface is what accepts user queries, routes request to the application backend, and displays generated responses. If you\u2019re working with client applications on mobile or desktop, .NET MAUI would be a good option for this component as well. <\/p>\n<h4>Application backend<\/h4>\n<p>The application backend is an <a href=\"https:\/\/learn.microsoft.com\/aspnet\/core\/fundamentals\/minimal-apis\/overview?view=aspnetcore-7.0\">ASP.NET Core Minimal API<\/a>. The backend hosts the Blazor static web application and what orchestrates the interactions among the different services. Services used in this application include:<\/p>\n<ul>\n<li><strong>Azure Cognitive Search<\/strong> &#8211; indexes documents from the data stored in an Azure Storage Account. This makes the documents searchable. <\/li>\n<li><strong>Azure OpenAI Service<\/strong> &#8211; provides the ChatGPT models to generate responses. Additionally, <a href=\"https:\/\/learn.microsoft.com\/semantic-kernel\/whatissk\">Semantic Kernel<\/a> is used in conjunction with the Azure OpenAI Service to orchestrate the more complex AI workflows.<\/li>\n<li><strong><a href=\"https:\/\/learn.microsoft.com\/azure\/azure-cache-for-redis\/cache-overview\">Azure Redis Cache<\/a><\/strong> &#8211; caches responses. Doing so reduces latency when generating responses to similar questions and helps manage costs because you don\u2019t have to make another request to Azure OpenAI Service.<\/li>\n<\/ul>\n<h4>Resource provisioning and developer environments<\/h4>\n<p>With all the services mentioned, the process of getting started may seem difficult. However, we\u2019ve streamlined that process using the <a href=\"https:\/\/learn.microsoft.com\/azure\/developer\/azure-developer-cli\/overview\">Azure Developer CLI<\/a>. If you don\u2019t want to commit to installing any of the dependencies, we can help you there as well. Just open the repository in <a href=\"https:\/\/github.com\/features\/codespaces\">GitHub Codespaces<\/a> and use the Azure Developer CLI to provision your services. <\/p>\n<h3>Using ChatGPT on your documents<\/h3>\n<p>Now that you know the components that make up this app, let\u2019s see how they work together in the context of a chat.<\/p>\n<p>Before you chat with your documents, you\u2019ll want to have a knowledgebase you can query. Chances are, you might already have one. For this sample application, we\u2019ve kept it simple. There are a set of PDF documents in the application\u2019s data directory. To load them into Azure Storage and index them in Azure Cognitive Search, we\u2019ve built a C# console application.<\/p>\n<p><img decoding=\"async\" src=\".\/app-architecture-build-kb.svg\" alt=\"App architecture showing knowledgebase building components\" \/><\/p>\n<p>The C# console application does the following:<\/p>\n<ol>\n<li>Uses Azure Form Recognizer to extract the text from each of the documents.<\/li>\n<li>Splits the documents into smaller excerpts. (chunking)<\/li>\n<li>Creates a new PDF document for each of the excerpts.<\/li>\n<li>Uploads the excerpt to an Azure Storage Account.<\/li>\n<li>Creates an index in Azure Cognitive Search.<\/li>\n<li>Adds the documents to the Azure Cognitive Search index.<\/li>\n<\/ol>\n<p>Why are the PDFs split into chunks? <\/p>\n<p>OpenAI models like ChatGPT have token limits. For more information on token limits see the <a href=\"https:\/\/learn.microsoft.com\/azure\/cognitive-services\/openai\/quotas-limits#quotas-and-limits-reference\">Azure OpenAI Service quotas and limits reference guide<\/a>. <\/p>\n<p>In this example, the knowledgebase building process is manual. However, based on your needs you may opt to run this as an event-driven job whenever a new document is added to the Azure Storage Account or in batches as a background job. <\/p>\n<p>Another pattern worth mentioning involves the use of embeddings to encode semantic information about your data. These embeddings are typically stored in vector databases. For a quick introduction to embeddings, see the <a href=\"https:\/\/learn.microsoft.com\/azure\/cognitive-services\/openai\/concepts\/understand-embeddings\">Azure OpenAI embeddings documentation<\/a>. That pattern is not the main one used in this sample and beyond the scope of this post. However, in future posts, we&#8217;ll cover those topics in more detail, so stay tuned! <\/p>\n<h3>Chat with your data<\/h3>\n<p>Once you have your knowledgebase set up, it\u2019s time to chat with it. <\/p>\n<p><img decoding=\"async\" src=\".\/dotnet-openai-enterprise-chat-app.png\" alt=\"Image of a question being asked on ChatGPT .NET web app\" \/><\/p>\n<p>Querying the knowledgebase starts off with the user entering a question in the Blazor web app. The user query is then routed to the ASP.NET Core Minimal Web API.<\/p>\n<p>Inside the web api, the <code>chat<\/code> endpoint handles the request. <\/p>\n<pre><code class=\"language-csharp\">api.MapPost(\"chat\", OnPostChatAsync);<\/code><\/pre>\n<p>To handle the request, we apply a pattern known as Retrieval Augmented Generation which does the following:<\/p>\n<ol>\n<li>Queries the knowledgebase for relevant documents<\/li>\n<li>Uses the relevant documents as context to generate a response<\/li>\n<\/ol>\n<h4>Querying the knowledgebase<\/h4>\n<p><img decoding=\"async\" src=\".\/app-architecture-query.svg\" alt=\"App architecture showing knowledgebase querying components\" \/><\/p>\n<p>The knowledgebase is queried using Azure Cognitive Search. Azure Cognitive Search though doesn\u2019t understand the natural language provided by the user. Fortunately, we can use ChatGPT to help translate the natural language into a query. <\/p>\n<p>Using Semantic Kernel, we create a method that defines a prompt template and adds the chat history and user question as additional context to generate an Azure Cognitive Search query.<\/p>\n<pre><code class=\"language-csharp\">private ISKFunction CreateQueryPromptFunction(ChatTurn[] history)\n{\n    var queryPromptTemplate = \"\"\"\n        Below is a history of the conversation so far, and a new question asked by the user that needs to be answered by searching in a knowledge base about employee healthcare plans and the employee handbook.\n        Generate a search query based on the conversation and the new question.\n        Do not include cited source filenames and document names e.g info.txt or doc.pdf in the search query terms.\n        Do not include any text inside [] or &lt;&lt;&gt;&gt; in the search query terms.\n        If the question is not in English, translate the question to English before generating the search query.\n\n        Chat History:\n        {{$chat_history}}\n\n        Question:\n        {{$question}}\n\n        Search query:\n        \"\"\";\n\n    return _kernel.CreateSemanticFunction(queryPromptTemplate,\n        temperature: 0,\n        maxTokens: 32,\n        stopSequences: new[] { \"\\n\" });\n}<\/code><\/pre>\n<p>That method is then used to compose the query generation prompt. <\/p>\n<pre><code class=\"language-csharp\">var queryFunction = CreateQueryPromptFunction(history);\nvar context = new ContextVariables();\ncontext[\"chat_history\"] = history.GetChatHistoryAsText();\ncontext[\"question\"] = userQuestion;<\/code><\/pre>\n<p>When you run the Semantic Kernel function, it provides the composed prompt to the Azure OpenAI Service ChatGPT model which generates the query. <\/p>\n<pre><code class=\"language-csharp\">var query = await _kernel.RunAsync(context, cancellationToken, queryFunction);<\/code><\/pre>\n<p>Given the question <em>\u201cWhat is included in my Northwind Health Plus plan that is not in standard?\u201d<\/em>, the generated query might look like <em>Northwind Health Plus plan coverage<\/em><\/p>\n<p>Once your query is generated, use the Azure Cognitive Search client to query the index containing your documents.<\/p>\n<pre><code class=\"language-csharp\">var documentContents = await _searchClient.QueryDocumentsAsync(query.Result, overrides, cancellationToken);<\/code><\/pre>\n<p>At this point, Azure Cognitive Search will return results containing the documents most relevant to your query. Results might look like the following:<\/p>\n<blockquote>\n<p>Northwind_Health_Plus_Benefits_Details-108.pdf: You must provide Northwind Health Plus with a copy of the EOB for the Primary Coverage, as well as a copy of the claim that you submitted to your Primary Coverage. This will allow us to determine the benefits that are available to you under Northwind Health Plus. It is important to note that Northwind Health Plus does not cover any expenses that are considered to be the responsibility of the Primary Coverage.  <\/p>\n<p>Benefit_Options-3.pdf: The plans also cover preventive care services such as mammograms, colonoscopies, and other cancer screenings. Northwind Health Plus offers more comprehensive coverage than Northwind Standard. This plan offers coverage for emergency services, both in-network and out-of-network, as well as mental health and substance abuse coverage. Northwind Standard does not offer coverage for emergency services, mental health and substance abuse coverage, or out-of-network services <\/p>\n<\/blockquote>\n<h4>Generating a response<\/h4>\n<p>Now that you have documents with relevant information to help answer the user question, it\u2019s time to use them to generate an answer. <\/p>\n<p><img decoding=\"async\" src=\".\/app-architecture-generate-response.svg\" alt=\"App architecture showing response generation components\" \/><\/p>\n<p>We start off by using Semantic Kernel to create a function that composes a prompt containing chat history, documents, and follow-up questions. <\/p>\n<pre><code class=\"language-csharp\">private const string AnswerPromptTemplate = \"\"\"\n    &lt;|im_start|&gt;system\n    You are a system assistant who helps the company employees with their healthcare plan questions, and questions about the employee handbook. Be brief in your answers.\n    Answer ONLY with the facts listed in the list of sources below. If there isn't enough information below, say you don't know. Do not generate answers that don't use the sources below. If asking a clarifying question to the user would help, ask the question.\n    For tabular information return it as an html table. Do not return markdown format.\n    Each source has a name followed by colon and the actual information, always include the full path of source file for each fact you use in the response. Use square brakets to reference the source. Don't combine sources, list each source separately.\n    ### Examples\n    Sources:\n    info1.txt: deductibles depend on whether you are in-network or out-of-network. In-network deductibles are $500 for employees and $1000 for families. Out-of-network deductibles are $1000 for employees and $2000 for families.\n    info2.pdf: Overlake is in-network for the employee plan.\n    reply: In-network deductibles are $500 for employees and $1000 for families [info1.txt][info2.pdf] and Overlake is in-network for the employee plan [info2.pdf].\n    ###\n    {{$follow_up_questions_prompt}}\n    {{$injected_prompt}}\n    Sources:\n    {{$sources}}\n    &lt;|im_end|&gt;\n    {{$chat_history}}\n    \"\"\";\n\nprivate const string FollowUpQuestionsPrompt = \"\"\"\n    Generate three very brief follow-up questions that the user would likely ask next about their healthcare plan and employee handbook.\n    Use double angle brackets to reference the questions, e.g. &lt;&lt;Are there exclusions for prescriptions?&gt;&gt;.\n    Try not to repeat questions that have already been asked.\n    Only generate questions and do not generate any text before or after the questions, such as 'Next Questions'\n    \"\"\";<\/code><\/pre>\n<p>Are we hard-coding answers in the prompt? <\/p>\n<p>We\u2019re not. The examples in the prompt serve as guidelines for the model to generate the answer. This is known as few-shot learning. <\/p>\n<pre><code class=\"language-csharp\">private ISKFunction CreateAnswerPromptFunction(string answerTemplate, RequestOverrides? overrides) =&gt;\n    _kernel.CreateSemanticFunction(answerTemplate,\n        temperature: overrides?.Temperature ?? 0.7,\n        maxTokens: 1024,\n        stopSequences: new[] { \"&lt;|im_end|&gt;\", \"&lt;|im_start|&gt;\" });\n\nISKFunction answerFunction;\nvar answerContext = new ContextVariables();\n\nanswerContext[\"chat_history\"] = history.GetChatHistoryAsText();\nanswerContext[\"sources\"] = documentContents;\nanswerContext[\"follow_up_questions_prompt\"] = ReadRetrieveReadChatService.FollowUpQuestionsPrompt;\n\nanswerFunction = CreateAnswerPromptFunction(ReadRetrieveReadChatService.AnswerPromptTemplate, overrides);\n\nprompt = ReadRetrieveReadChatService.AnswerPromptTemplate;<\/code><\/pre>\n<p>When you run the Semantic Kernel function, it provides the composed prompt to the Azure OpenAI Service ChatGPT model which generates the response. <\/p>\n<pre><code class=\"language-csharp\">var ans = await _kernel.RunAsync(answerContext, cancellationToken, answerFunction);<\/code><\/pre>\n<p>After some formatting, the answer is returned to the web app and displayed. The result might look similar to the following:<\/p>\n<p><img decoding=\"async\" src=\".\/chat-responses.png\" alt=\"Image displaying responses of ChatGPT generated responses in .NET web app\" \/><\/p>\n<p>In order to build more trust in the responses, the response includes citations, the full prompt used to generate the response, and supporting content containing the documents from the search results. <\/p>\n<h2>Build your own intelligent apps<\/h2>\n<p>We\u2019re excited for the future of intelligent applications in .NET.<\/p>\n<p>Check out the application source code on GitHub and use it as a template to start building intelligent applications with your own data. <\/p>\n<p><div  class=\"d-flex justify-content-center\"><a class=\"cta_button_link btn-primary mb-24\" href=\"https:\/\/aka.ms\/dotnet-ai-app\" target=\"_blank\">Build Your Own .NET Intelligent App<\/a><\/div><\/p>\n<h2>We want to hear from you!<\/h2>\n<p>Are you interested in building or currently building intelligent apps? Take a few minutes to complete this survey.<\/p>\n<p><div  class=\"d-flex justify-content-center\"><a class=\"cta_button_link btn-primary mb-24\" href=\"https:\/\/aka.ms\/dotnet-build-oai-survey\" target=\"_blank\">Take the survey<\/a><\/div><\/p>\n<h2>Additional resources<\/h2>\n<ul>\n<li><a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/getting-started-azure-openai-dotnet\/\">Get started with OpenAI in .NET<\/a><\/li>\n<li><a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/get-started-with-open-ai-completions-with-dotnet\/\">Get Started with OpenAI Completions with .NET<\/a><\/li>\n<li><a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/gpt-prompt-engineering-openai-azure-dotnet\/\">Level up your GPT game with prompt engineering<\/a><\/li>\n<li><a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/get-started-chatgpt-azure-dotnet\/\">Get started with ChatGPT in .NET<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Learn how you can build intelligent apps and unleash the full potential of AI in your .NET applications using ChatGPT.<\/p>\n","protected":false},"author":26108,"featured_media":45828,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[685,7781,328,7509,327,7251,756],"tags":[7189,568,7742,7724],"class_list":["post-45823","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet","category-ai","category-aiml","category-aspnetcore","category-azure","category-blazor","category-csharp","tag-machinelearning","tag-ai","tag-chatgpt","tag-openai"],"acf":[],"blog_post_summary":"<p>Learn how you can build intelligent apps and unleash the full potential of AI in your .NET applications using ChatGPT.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/45823","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/26108"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=45823"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/45823\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/45828"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=45823"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=45823"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=45823"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}