To that end, this post focuses on building a simple console-based .NET chat application from the ground up, with minimal dependencies and minimal fuss. The end goal is to be able to ask questions and get answers not only based on the data on which our model was trained, but also on additional data supplied dynamically. Along the way, every code sample shown is a complete application, so you can just copy-and-paste it into a To begin, make sure you have .NET 8 installed, and create a simple console app (.NET 6, .NET 7, and .NET Framework will also work as well, just with a few tweaks to the project file):<\/p>\n This creates a new directory We could choose to reference specific AI-related packages, like Azure.AI.OpenAI<\/a>, but I’ve instead turned to Semantic Kernel<\/a> (SK) as a way to simplify various interactions and more easily swap in and out different implementations in order to more quickly experiment. SK provides a set of libraries that makes it easier to work with Large Language Models (LLMs). It provides abstractions for various AI concepts so that you can code to the abstraction and more easily substitute different implementations. It provides many concrete implementations of those abstractions, wrapping a multitude of other SDKs. It provides support for planning and orchestration, such that you can ask AI to create a plan for how to achieve a certain goal. It provides support for plug-ins. And much more. We’ll touch on a variety of those aspects throughout this post, but I’m primarily using SK for its abstractions.<\/p>\n While for the purposes of this post I’ve tried to keep dependencies to a minimum, there’s one more I can’t avoid: you need access to an LLM. The easiest way to get access is via either OpenAI<\/a> or Azure OpenAI<\/a>. For this post, I’m using OpenAI, but switching to use Azure OpenAI instead requires changing just one line in each of the samples. You’ll need three pieces of information for the remainder of the post:<\/p>\n With that out of the way, we can dive in. Believe it or not, we can create a simple chat app in just a few lines code. Copy-and-paste this into your To avoid accidentally leaking my API key (which needs to be protected like you would a password) to the world in this post, I’ve stored it in an environment variable. Thus, I read in the API key and endpoint via The The “function” in this example is just whatever the user typed, e.g. if the user typed “What color is the sky?”, that’s the function, asking of the LLM “What color is the sky?”, since that’s what we passed to When that function is invoked, it will render the prompt, filling in those placeholders by invoking the registered We’re making good progress: in just a few lines of code, we’ve been able to create a simple chat agent to which we can repeatedly pose questions and get answers, and we’ve been able to provide additional information in the prompt to help it answer questions it would have otherwise been unable to answer. However, we’ve also created a chat agent with no memory, such that it has no concept of things previously discussed:\n These LLMs are stateless. To address the lack of memory, we need to keep track of our chat history and send it back as part of the prompt on each request. We could do so manually, rendering it into the prompt ourselves, or we could rely on SK to do it for us (and it can rely on the underlying clients for Azure OpenAI, OpenAI, or whatever other chat service is plugged in). This does the latter, getting the registered With that chat history rendered into an appropriate prompt, we then get back much more satisfying results:\n In a real implementation, you’d need to pay attention to many other details, like the fact that all of these language models today have limits on the amount of data they’re able to process (the “context window”). The model I’m using has a context window of ~16,000 tokens (a “token” is the unit at which an LLM operates, sometimes a whole word, sometimes a portion of a word, sometimes a single character), plus there’s a per-token cost associated with every request\/response, so once this graduates from experimentation to “let’s put this into production,” we’d need to start paying a lot more attention to things like how much data is actually in the chat history, clearing out portions of it, etc.<\/p>\n We can also add a tiny bit of code to help make the interaction feel snappier. These LLMs work based on generating the next token in the response, so although up until now we’ve only been printing out the response when the whole thing has arrived, we can actually stream the results so that we print out portions of the response as it’s available. This is exposed in SK via Now when we run this, we can see the response streaming in:\nProgram.cs<\/code> file, run it, play with it, and then copy-and-paste into your real application, where you can refine and augment it to your heart’s content.<\/p>\nLet’s Get Chattin’<\/h2>\n
dotnet new console -o chatapp\r\ncd chatapp<\/code><\/pre>\nchatapp<\/code> and populates it with two files: chatapp.csproj<\/code> and Program.cs<\/code>. We then need to bring in one NuGet package: Microsoft.SemanticKernel<\/code>.<\/p>\ndotnet add package Microsoft.SemanticKernel<\/code><\/pre>\n\n
gpt-3.5-turbo-0125<\/code>, which as of this writing has a context window of ~16K tokens. We’ll talk more about what this is later. Note that if you’re using Azure OpenAI instead of OpenAI, you won’t refer to the model by name; instead, you’ll create a deployment of that model and refer to the deployment name.\ntext-embedding-3-small<\/code>.<\/li>\n<\/ul>\nProgram.cs<\/code>:<\/p>\nusing Microsoft.SemanticKernel;\r\n\r\nstring apikey = Environment.GetEnvironmentVariable(\"AI:OpenAI:APIKey\")!;\r\n\r\n\/\/ Initialize the kernel\r\nKernel kernel = Kernel.CreateBuilder()\r\n .AddOpenAIChatCompletion(\"gpt-3.5-turbo-0125\", apikey)\r\n .Build();\r\n\r\n\/\/ Q&A loop\r\nwhile (true)\r\n{\r\n Console.Write(\"Question: \");\r\n Console.WriteLine(await kernel.InvokePromptAsync(Console.ReadLine()!));\r\n Console.WriteLine();\r\n}<\/code><\/pre>\nGetEnvironmentVariable<\/code>. I then create a new “kernel” with the SK APIs, asking it to add into the kernel an OpenAI chat completion service. The Microsoft.SemanticKernel<\/code> package we pulled in earlier includes references to client support for both OpenAI and Azure OpenAI, so we don’t need anything additional to be able to talk to these services. And with that configured, we can now run our chat app (dotnet run<\/code>), typing out questions and getting answers back from the service:\n![\"Simple](\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/09\/SimpleQA.png\")
<\/p>\nawait kernel.InvokePromptAsync(Console.ReadLine()!)<\/code> expression in there is the entirety of the interaction with the LLM. This reads in the user’s question and sends it off to the LLM, getting back a string<\/code> response. SK supports multiple kinds of functions that can be invoked, including prompt functions (text-based interactions with AI) and normal .NET methods that can do anything C# code can do. The invocation of these functions can be issued directly by the consumer, as we’re doing here, but they can also be invoked as part of a “plan”: you supply a set of functions, each of which is capable of accomplishing something, and you ask the LLM for a plan for how to use one or more of those functions to achieve some described goal… SK can then invoke the functions according to the plan (we’ll see an example of that later). Some models also suppport “function calling”, which we’ll see in action later; this is also something SK simplifies.<\/p>\nInvokePromptAsync<\/code>. We can make this function nature a bit clearer by separating the function out into its own entity, via the CreateFunctionFromPrompt<\/code> method, and then reusing that one function repeatedly. In doing so, we’re no longer creating a new function per user input, and thus need some way to parameterize the created function with the user’s input. For that, SK includes support for prompt templates, where you supply the prompt but with placeholders that SK will fill in based on the variables and functions available to it. For example, if I run the previous sample again, and this time ask for the current time, the LLM is unable to provide me with an answer:\n![\"LLM](\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/09\/LackOfTime.png\")
\nHowever, if we expect such questions, we can proactively provide the LLM with the information it needs as part of the prompt. Here I’ve registered with the kernel a function that returns the current date and time. Then I’ve created a prompt function, with a prompt template that will invoke this function as part of rendering the prompt, and that will also include the value of the $input<\/code> variable (any number of arbitrarily-named arguments can be supplied via a KernelArguments<\/code> dictionary, and I’ve simply chosen to name one “input”). Functions are grouped into “plugins”:<\/p>\nusing Microsoft.SemanticKernel;\r\n\r\nstring apikey = Environment.GetEnvironmentVariable(\"AI:OpenAI:APIKey\")!;\r\n\r\n\/\/ Initialize the kernel\r\nKernel kernel = Kernel.CreateBuilder()\r\n .AddOpenAIChatCompletion(\"gpt-3.5-turbo-0125\", apikey)\r\n .Build();\r\n\r\n\/\/ Create the prompt function as part of a plugin and add it to the kernel.\r\n\/\/ These operations can be done separately, but helpers also enable doing\r\n\/\/ them in one step.\r\nkernel.ImportPluginFromFunctions(\"DateTimeHelpers\",\r\n[\r\n kernel.CreateFunctionFromMethod(() => $\"{DateTime.UtcNow:r}\", \"Now\", \"Gets the current date and time\")\r\n]);\r\n\r\nKernelFunction qa = kernel.CreateFunctionFromPrompt(\"\"\"\r\n The current date and time is {{ datetimehelpers.now }}.\r\n {{ $input }}\r\n \"\"\");\r\n\r\n\/\/ Q&A loop\r\nvar arguments = new KernelArguments();\r\nwhile (true)\r\n{\r\n Console.Write(\"Question: \");\r\n arguments[\"input\"] = Console.ReadLine();\r\n Console.WriteLine(await qa.InvokeAsync(kernel, arguments));\r\n Console.WriteLine();\r\n}<\/code><\/pre>\nNow<\/code> function and substituting its result into the prompt. Now when I ask the same question, the answer is more satisfying:\n![\"LLM](\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/09\/SuccessfulTime.png\")
<\/p>\nA Trip Down Memory Lane<\/h2>\n
![\"LLM](\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/09\/NoMemory.png\")
<\/p>\nIChatCompletionService<\/code>, creating a new chat (which is essentially just a list of all messages added to it), and then not only issuing requests and printing out responses, but also storing both into the chat history.<\/p>\nusing Microsoft.SemanticKernel;\r\nusing Microsoft.SemanticKernel.ChatCompletion;\r\n\r\nstring apikey = Environment.GetEnvironmentVariable(\"AI:OpenAI:APIKey\")!;\r\n\r\n\/\/ Initialize the kernel\r\nKernel kernel = Kernel.CreateBuilder()\r\n .AddOpenAIChatCompletion(\"gpt-3.5-turbo-0125\", apikey)\r\n .Build();\r\n\r\n\/\/ Create a new chat\r\nIChatCompletionService ai = kernel.GetRequiredService<IChatCompletionService>();\r\nChatHistory chat = new(\"You are an AI assistant that helps people find information.\");\r\n\r\n\/\/ Q&A loop\r\nwhile (true)\r\n{\r\n Console.Write(\"Question: \");\r\n chat.AddUserMessage(Console.ReadLine()!);\r\n\r\n var answer = await ai.GetChatMessageContentAsync(chat);\r\n chat.AddAssistantMessage(answer.Content!);\r\n Console.WriteLine(answer);\r\n\r\n Console.WriteLine();\r\n}<\/code><\/pre>\n![\"LLM](\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2023\/09\/WithMemory.png\")
<\/p>\nIAsyncEnumerable<T><\/code>, making it conventient to work with via await foreach<\/code> loops.<\/p>\nusing Microsoft.SemanticKernel;\r\nusing Microsoft.SemanticKernel.ChatCompletion;\r\nusing System.Text;\r\n\r\nstring apikey = Environment.GetEnvironmentVariable(\"AI:OpenAI:APIKey\")!;\r\n\r\n\/\/ Initialize the kernel\r\nKernel kernel = Kernel.CreateBuilder()\r\n .AddOpenAIChatCompletion(\"gpt-3.5-turbo-0125\", apikey)\r\n .Build();\r\n\r\n\/\/ Create a new chat\r\nIChatCompletionService ai = kernel.GetRequiredService<IChatCompletionService>();\r\nChatHistory chat = new(\"You are an AI assistant that helps people find information.\");\r\nStringBuilder builder = new();\r\n\r\n\/\/ Q&A loop\r\nwhile (true)\r\n{\r\n Console.Write(\"Question: \");\r\n chat.AddUserMessage(Console.ReadLine()!);\r\n\r\n builder.Clear();\r\n await foreach (StreamingChatMessageContent message in ai.GetStreamingChatMessageContentsAsync(chat))\r\n {\r\n Console.Write(message);\r\n builder.Append(message.Content);\r\n }\r\n Console.WriteLine();\r\n chat.AddAssistantMessage(builder.ToString());\r\n\r\n Console.WriteLine();\r\n}<\/code><\/pre>\n