{"id":53146,"date":"2024-08-13T11:00:00","date_gmt":"2024-08-13T18:00:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/dotnet\/?p=53146"},"modified":"2024-08-15T08:53:50","modified_gmt":"2024-08-15T15:53:50","slug":"azure-ai-model-catalog-dotnet-inference-sdk","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/azure-ai-model-catalog-dotnet-inference-sdk\/","title":{"rendered":"Introducing the Azure AI Inference SDK: Access More AI Models with the Azure AI Model Catalog"},"content":{"rendered":"<p>AI models are constantly evolving and improving, but keeping up with the latest developments can be challenging.<\/p>\n<p>That&#8217;s why we&#8217;re introducing the Azure AI Inference SDK for .NET.<\/p>\n<p>This SDK lets you easily access and use a wide range of AI models from the Azure AI Model Catalog for inference tasks like chat, so you can seamlessly integrate AI into your applications that meet your needs.<\/p>\n<h2>What is the Azure AI Model Catalog?<\/h2>\n<p><img decoding=\"async\" src=\".\/azure-ai-model-catalog.png\" alt=\"Image displaying models in the Azure AI Model Catalog\" \/><\/p>\n<p>The Model Catalog in Azure AI Studio makes it easy to browse through various AI models and deploy them.<\/p>\n<p>Models from the catalog can be deployed to Managed Compute or as a Serverless API.<\/p>\n<p>Some key features include:<\/p>\n<ul>\n<li><strong>Model Availability<\/strong>: The model catalog features a diverse collection of models from providers such as Microsoft, Azure OpenAI, Mistral, Meta, and Cohere. This ensures you can find the right model to satisfy your requirements.<\/li>\n<li><strong>Easy to deploy<\/strong>: Serverless API deployments remove the complexity about hosting and provisioning the hardware to run cutting edge models. When deploying models with serverless API, you don&#8217;t need quota to host them and you are billed per token.<\/li>\n<li><strong>Responsible AI Built-In<\/strong>: Safety is a priority. Language models from the catalog come with default configurations of Azure AI Content Safety moderation filters which detect harmful content.<\/li>\n<\/ul>\n<p>For more details, see the <a href=\"https:\/\/learn.microsoft.com\/azure\/ai-studio\/how-to\/model-catalog-overview\">Azure AI Model Catalog documentation<\/a>.<\/p>\n<h2>Get Started<\/h2>\n<ol>\n<li>Deploy a model like <a href=\"https:\/\/ai.azure.com\/explore\/models?selectedCollection=phi&amp;tid=72f988bf-86f1-41af-91ab-2d7cd011db47\">Phi-3<\/a>. For more details, see the <a href=\"https:\/\/learn.microsoft.com\/azure\/ai-studio\/how-to\/deploy-models-serverless?tabs=azure-ai-studio\">Azure AI Model Catalog deployment documentation<\/a>.<\/li>\n<li>Create a C# console application and install the <a href=\"https:\/\/www.nuget.org\/packages\/Azure.AI.Inference\/\">Azure.AI.Inference<\/a> SDK from NuGet.<\/li>\n<li>Add the following code to your application to start making requests to your model service. Make sure to replace your key and endpoint with those provided with your deployment.<\/li>\n<\/ol>\n<pre><code class=\"language-csharp\">var key = \"YOUR-MODEL-API-KEY\";\nvar endpoint = \"YOUR-MODEL-ENDPOINT\";\n\nvar chatClient = new ChatCompletionsClient(\n    new Uri(endpoint), \n    new Azure.AzureKeyCredential(key));\n\nvar chatHistory = new List&lt;ChatRequestMessage&gt;()\n{\n    new ChatRequestSystemMessage(\"You are a helpful assistant that knows about AI.\")\n};\n\nConsole.WriteLine($\"System: {\n    chatHistory\n    .Where(x =&gt; x.GetType() == typeof(ChatRequestSystemMessage))\n    .Select(x =&gt; ((ChatRequestSystemMessage)x).Content)\n    .First()}\");\n\nwhile(true)\n{\n    Console.Write(\"You: \");\n    var userMessage = Console.ReadLine();\n\n    \/\/ Exit loop\n    if (userMessage.StartsWith(\"\/q\"))\n    {\n        break;\n    }\n\n    chatHistory.Add(new ChatRequestUserMessage(userMessage));\n\n    ChatCompletions? response = await chatClient.CompleteAsync(chatHistory);\n    ChatResponseMessage? assistantMessage = response.Choices.First().Message;\n    chatHistory.Add(new ChatRequestAssistantMessage(assistantMessage));\n\n    Console.WriteLine($\"Assistant: {assistantMessage.Content}\");\n}<\/code><\/pre>\n<p><img decoding=\"async\" src=\".\/azure-ai-inference-sdk-sample-console-output.png\" alt=\"Sample console output of a chat with a model from Azure AI model catalog\" \/><\/p>\n<p>For more details, see the <a href=\"https:\/\/aka.ms\/azsdk\/azure-ai-inference\/csharp\/reference\">Azure AI Model Inference API documentation<\/a>.<\/p>\n<h2>Conclusion<\/h2>\n<p>We&#8217;re excited to see what you build! Try out the Azure AI Inference SDK and give us feedback.  <\/p>\n","protected":false},"excerpt":{"rendered":"<p>Announcing the Azure AI Inference SDK for .NET! This SDK provides easy access to a comprehensive suite of AI models in the Azure AI Model Catalog for inference tasks like chat, enabling you to effortlessly incorporate AI into your applications that align with your requirements.<\/p>\n","protected":false},"author":26108,"featured_media":53214,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[685,7781,327,688],"tags":[568,37,7862],"class_list":["post-53146","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet","category-ai","category-azure","category-machine-learning","tag-ai","tag-azure","tag-azureai"],"acf":[],"blog_post_summary":"<p>Announcing the Azure AI Inference SDK for .NET! This SDK provides easy access to a comprehensive suite of AI models in the Azure AI Model Catalog for inference tasks like chat, enabling you to effortlessly incorporate AI into your applications that align with your requirements.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/53146","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/26108"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=53146"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/53146\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/53214"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=53146"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=53146"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=53146"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}