{"id":2041,"date":"2024-03-06T10:17:23","date_gmt":"2024-03-06T18:17:23","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/semantic-kernel\/?p=2041"},"modified":"2024-03-27T16:02:50","modified_gmt":"2024-03-27T23:02:50","slug":"architecting-ai-apps-with-semantic-kernel","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/agent-framework\/architecting-ai-apps-with-semantic-kernel\/","title":{"rendered":"Architecting AI Apps with Semantic Kernel"},"content":{"rendered":"<p>With Semantic Kernel, you can easily build agents that can call your existing code. This power lets you automate your business processes with models from <a href=\"https:\/\/platform.openai.com\/docs\/\">OpenAI<\/a>,\u00a0<a href=\"https:\/\/azure.microsoft.com\/products\/cognitive-services\/openai-service\/\">Azure OpenAI<\/a>,\u00a0<a href=\"https:\/\/huggingface.co\/\">Hugging Face<\/a>, and more! We often get asked though, \u201cHow do I architect my solution?\u201d and \u201cHow does it <em>actually<\/em> work?\u201d<\/p>\n<p>In this blog post, we\u2019ll break apart the key components of Semantic Kernel (Plugins, Planners, and Personas), and show how you can combine Semantic Kernel with your existing apps and services.<\/p>\n<h2><span style=\"font-size: 24pt;\">What is a plugin?<\/span><a href=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2024\/03\/Plugins-1.png\"><img decoding=\"async\" class=\"wp-image-2053 size-full aligncenter\" src=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2024\/03\/Plugins-1.png\" alt=\"Image Plugins 1\" width=\"600\" height=\"601\" srcset=\"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/Plugins-1.png 600w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/Plugins-1-300x300.png 300w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/Plugins-1-150x150.png 150w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/Plugins-1-24x24.png 24w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/Plugins-1-48x48.png 48w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/Plugins-1-96x96.png 96w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/a><\/h2>\n<p><strong>Plugins<\/strong> are the fundamental building blocks of Semantic Kernel. If you have already used plugins from ChatGPT, Bing, or Microsoft 365, you\u2019re already familiar with. With plugins, you can encapsulate capabilities into a single unit of functionality that can be used by an AI. Because plugins are the same across all platforms, you can bring these same plugins to your own AI powered applications with Semantic Kernel.<\/p>\n<p>Plugins are highly adaptable; they can consist of both native code <em>and<\/em> requests to AI services via prompts. Examples include:<\/p>\n<ul>\n<li><strong>Web search plugin:<\/strong> Gives the AI the ability to search the web for current information that might not exist in its training data.<\/li>\n<li><strong>CRM plugin<\/strong>: Allows an AI to retrieve information about customers and make adjustments as new information comes in.<\/li>\n<li><strong>Email plugin<\/strong>: Let\u2019s an AI send and receive emails on behalf of a user.<\/li>\n<li><strong>Inventory plugins<\/strong>: Gives an AI the ability to see what is in stock so it can give recommendations to employees and customers alike.<\/li>\n<li><strong>Image generation plugin<\/strong>: You can augment an AI with capabilities from other AI models, like a text-to-image service.<\/li>\n<li><strong>Microsoft 365 Plugin:<\/strong> Allows an AI to use Microsoft 365 applications, such as Microsoft Word so that it can edit documents on behalf of a user.<\/li>\n<\/ul>\n<h2><span style=\"font-size: 24pt;\">Using plugins with Semantic Kernel<\/span><\/h2>\n<p>What makes Semantic Kernel powerful is its ability to combine a user\u2019s request with your available plugins. In the following example, a User sends a request to an agent. The agent then relays the request to Semantic Kernel where it\u2019s combined with the power of plugins and a persona.<\/p>\n<p><img decoding=\"async\" class=\"wp-image-2054 size-full aligncenter\" src=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2024\/03\/Semantic-Kernel-overview-1.png\" alt=\"Image Semantic Kernel overview 1\" width=\"600\" height=\"1119\" srcset=\"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/Semantic-Kernel-overview-1.png 600w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/Semantic-Kernel-overview-1-161x300.png 161w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/Semantic-Kernel-overview-1-549x1024.png 549w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/p>\n<p>Within each plugin, you can give actions to the agent. These actions could be API calls (e.g., get data from Microsoft graph), access to memory (e.g., get answers from documents) or native functions (e.g., get the current time).<\/p>\n<p>With a persona, you can dictate <em>how<\/em> the agent should help the user. Is it snarky? Is it careful? Are there things it should or shouldn\u2019t do?<\/p>\n<p>Once these components are defined, Semantic Kernel can send a single prompt that <strong>combines<\/strong> the user request, plugins, and persona to an AI Model (OpenAI ChatGPT, AzureOpenAI, Gemini, Mistral AI, etc.) so it can plan the best way to achieve the user\u2019s goal. Finally, the agent relays the AI response back to the user.<\/p>\n<h2><span style=\"font-size: 24pt;\">How you could recreate Microsoft Word Copilot<\/span><\/h2>\n<p>You\u2019re likely familiar with Microsoft Word and other M365 productivity apps. Plugins are a key component to how the Copilots work in these apps. The following scenario focuses on how Word uses plugins to respond to a user\u2019s request to add a paragraph on the history of AI to an existing document. In this example, we\u2019ve given the Word Copilot access to plugins to search the web, retrieve previous memories about the document, and the ability to manipulate the doc.<\/p>\n<p><img decoding=\"async\" class=\"wp-image-2055 size-full aligncenter\" src=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2024\/03\/Semantic-Kernel-with-Word-2.png\" alt=\"Image Semantic Kernel with Word 2\" width=\"600\" height=\"1152\" srcset=\"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/Semantic-Kernel-with-Word-2.png 600w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/Semantic-Kernel-with-Word-2-156x300.png 156w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/Semantic-Kernel-with-Word-2-533x1024.png 533w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/p>\n<p>For this scenario, the AI can use these plugins to search the web for history about AI, recall other references about AI in the current doc, and add a new paragraph with the final content. As the AI model decides that it wants to use these plugins, it sends a request to Semantic Kernel to invoke the plugins.<\/p>\n<p>Once the AI is done performing the actions, it can relay a final response from the AI back to the user telling them that the paragraph has been added to the Word document.<\/p>\n<h2><span style=\"font-size: 24pt;\">Building your own Copilot with Semantic Kernel<\/span><\/h2>\n<p>If you want to build your own Copilot within your AI powered application, you can follow a similar pattern as the Microsoft 365 Copilots. The only difference is that you\u2019ll provide Semantic Kernel with your own custom plugins. In the following diagram, we demonstrate how you can create plugins that call your existing app service.<\/p>\n<p><img decoding=\"async\" class=\"wp-image-2056 size-full aligncenter\" src=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2024\/03\/App-server-1.png\" alt=\"Image App server 1\" width=\"600\" height=\"1320\" srcset=\"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/App-server-1.png 600w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/App-server-1-136x300.png 136w, https:\/\/devblogs.microsoft.com\/agent-framework\/wp-content\/uploads\/sites\/78\/2024\/03\/App-server-1-465x1024.png 465w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/p>\n<p>In the above example, the user asks the AI to create an infographic. To support this type of scenario, you would need to provide Semantic Kernel with an order plugin that can pull the relevant information from a database like Cosmos. You likely already have APIs that support these scenarios, so building these plugins should be easy!<\/p>\n<h2><span style=\"font-size: 24pt;\">Tell us about the plugins you build!<\/span><\/h2>\n<p>We\u2019re always interested in how developers are using AI to make their company more productive. With plugins, you can give AIs the ability to do nearly anything within your company. If you have an interesting scenario, feel free to share it with the rest of the community on the<a href=\"https:\/\/github.com\/microsoft\/semantic-kernel\/discussions\"> discussion boards<\/a> on GitHub!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>With Semantic Kernel, you can easily build agents that can call your existing code. This power lets you automate your business processes with models from OpenAI,\u00a0Azure OpenAI,\u00a0Hugging Face, and more! We often get asked though, \u201cHow do I architect my solution?\u201d and \u201cHow does it actually work?\u201d In this blog post, we\u2019ll break apart the [&hellip;]<\/p>\n","protected":false},"author":149071,"featured_media":2300,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[45,40],"class_list":["post-2041","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-semantic-kernel","tag-semantic-kernel-reference-architecture-ai","tag-tutorial"],"acf":[],"blog_post_summary":"<p>With Semantic Kernel, you can easily build agents that can call your existing code. This power lets you automate your business processes with models from OpenAI,\u00a0Azure OpenAI,\u00a0Hugging Face, and more! We often get asked though, \u201cHow do I architect my solution?\u201d and \u201cHow does it actually work?\u201d In this blog post, we\u2019ll break apart the [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/2041","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/users\/149071"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/comments?post=2041"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/2041\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media\/2300"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media?parent=2041"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/categories?post=2041"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/tags?post=2041"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}