{"id":1428,"date":"2023-11-02T15:31:42","date_gmt":"2023-11-02T22:31:42","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/semantic-kernel\/?p=1428"},"modified":"2024-01-10T13:40:39","modified_gmt":"2024-01-10T21:40:39","slug":"autogen-agents-meet-semantic-kernel","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/agent-framework\/autogen-agents-meet-semantic-kernel\/","title":{"rendered":"AutoGen Agents Meet Semantic Kernel"},"content":{"rendered":"<p>In this blog post, we show you how you can use Semantic Kernel with\u00a0<a href=\"https:\/\/github.com\/microsoft\/autogen\" target=\"_blank\" rel=\"noopener\">AutoGen<\/a>, a Microsoft Research project that shows the potential of using multiple agents together. With AutoGen, Microsoft research has shown that multiple agents can be better than one.<\/p>\n<p>To actually\u00a0<em>do<\/em>\u00a0work, however, agents within AutoGen need hooks into the real world so they can retrieve information, create content, and complete tasks. Without these capabilities, agents will simply talk to each other and waste thousands or tens-of-thousands of tokens.<\/p>\n<p><strong>This is where Semantic Kernel comes in. With Semantic Kernel, you can easily bring Plugins to agents so they can do more than just talk.<\/strong>\u00a0They can search the web, access company resources, and ultimately become productive parts of your company.<\/p>\n<blockquote><p>Plugins are important in the world of \u201cagents.\u201d Without them, it\u2019s impossible to get agents to complete real work.<\/p><\/blockquote>\n<p>Why would you want to work with agents? Used improperly, they can quickly consume thousands of tokens, but given the right tools (i.e., plugins) and a constrained enough task, multiple agents powered by gpt-3.5-turbo can rival the capabilities of gpt-4, all while using cheaper tokens with lower latency. While it\u2019s still early days for multi-agent systems, here at Microsoft we believe systems like AutoGen will substantially push the AI frontier forward.<\/p>\n<h2>How Agents Work<button class=\"linkicon\" title=\"\" aria-label=\"Copy Post URL\" data-id-href=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/?p=1428&amp;preview_id=1428&amp;preview_nonce=2755ad3571&amp;post_format=standard&amp;_thumbnail_id=1429&amp;preview=true#how-agents-work\" data-toggle=\"tooltip\" data-placement=\"right\" data-original-title=\"Copy Post URL\"><i class=\"fabric-icon fabric-icon--Link\"><\/i><\/button><\/h2>\n<div class=\"screenreader-text\" aria-atomic=\"true\" aria-live=\"polite\"><\/div>\n<p>Agents are based on the principle that diverse teams produce better outcomes. Greater diversity in opinion is facilitated by essentially making LLMs chat with each other. In doing so, they go beyond a\u00a0<a href=\"https:\/\/arxiv.org\/abs\/2201.11903\" target=\"_blank\" rel=\"noopener\">\u201cChain of Thought\u201d<\/a>-style monologue and instead involve seeding conversations between LLMs that have different Personas.<\/p>\n<blockquote><p>&#8220;In addition to diversity, there&#8217;s also specialization and modularity. Agents allow us to define AI copilots as specialized (i.e. &#8220;skilled agents&#8221;) on given goals, e.g. working on a specific area, looking after security, moderating content, etc. Better yet, agents can be combined. Modularity is important because it allows agent developers to focus on one area, rather than trying to create one do-it-all AI.&#8221; \u2014Devis Lucato, SK Founding Architect<\/p><\/blockquote>\n<p>For example, imagine one agent with a copywriter system prompt:<\/p>\n<pre class=\"prettyprint language-cs language-csharp\"><code class=\"language-cs language-csharp\">const string SystemMessageCopyWriter = @\"\r\nYou are a copywriter with ten years of experience and are known for a brevity and a dry humor.\r\nYou are laser-focused on the goal at hand. Don't waste time with chit chat.\r\nThe goal is to refine and decide on the best copy as an expert in the field.\";\r\n\r\nKernel kernel = Kernel.CreateBuilder().AddAzureOpenAIChatCompletion(\r\n                    MyModelId, MyEndpoint, MyApiKey)\r\n                .Build();\r\n\r\nvar chatCompletion = kernel.GetRequiredService&lt;IChatCompletionService&gt;();\r\n\r\nvar chatHistory = new ChatHistory(SystemMessageCopyWriter);\r\n\r\n\/\/ Add the user query to the chat history\r\nchatHistory.AddUserMessage(userMessage);\r\n\r\n\/\/ Finally, get the response from AI\r\nvar answer = await chatCompletion.GetChatMessageContentAsync(chatHistory);<\/code><\/pre>\n<p>Meanwhile, another agent can be given a different persona: an art director.<\/p>\n<pre class=\"prettyprint language-cs language-csharp prettyprinted\" tabindex=\"0\"><code class=\"language-cs language-csharp\"><span class=\"kwd\">var<\/span><span class=\"pln\"> systemMessageArtDirector <\/span><span class=\"pun\">=<\/span> <span class=\"str\">\"<\/span><span class=\"typ\">You<\/span><span class=\"pln\"> are an art director who has opinions about copywriting\r\nborn of a love <\/span><span class=\"kwd\">for<\/span> <span class=\"typ\">David<\/span> <span class=\"typ\">Ogilvy<\/span><span class=\"pun\">.<\/span> <span class=\"typ\">You<\/span><span class=\"str\">'<\/span><span class=\"pln\">re laser focused on the goal at hand<\/span><span class=\"pun\">.<\/span>\r\n<span class=\"typ\">Don<\/span><span class=\"str\">'<\/span><span class=\"pln\">t waste time with chit chat<\/span><span class=\"pun\">.<\/span> <span class=\"typ\">The<\/span><span class=\"pln\"> goal <\/span><span class=\"kwd\">is<\/span><span class=\"pln\"> to refine and decide on the best\r\ncopy <\/span><span class=\"kwd\">as<\/span><span class=\"pln\"> an expert <\/span><span class=\"kwd\">in<\/span><span class=\"pln\"> the field<\/span><span class=\"pun\">.<\/span><span class=\"str\">\"<\/span><\/code><\/pre>\n<p>Using either Autogen or Semantic Kernel, you can then get these two agents to converse with each other. For example, the following conversation was generated by the two agents after giving them the initial marketing prompt: \u201cmaps made out of egg cartons.\u201d<\/p>\n<blockquote><p><strong>Copywriter:<\/strong>\u00a0\u201cCrack open a world of sustainability with our egg-carton maps. Navigate the globe, one recycled egg carton at a time.\u201d<\/p>\n<p><strong>Art Director:<\/strong>\u00a0The concept is intriguing, but the copy could be more direct and impactful. Let\u2019s try: \u201cNavigate the globe sustainably with our recycled egg-carton maps. One carton, one step towards a greener world.\u201d This version emphasizes the product\u2019s unique selling point and its environmental benefits more clearly.<\/p>\n<p><strong>Copywriter:<\/strong>\u00a0\u201cRecycled Egg-Carton Maps: Chart a greener course for our planet, one carton at a time.\u201d<\/p>\n<p><strong>Art Director:<\/strong>\u00a0This is a strong revision. It\u2019s concise, direct, and clearly communicates the product\u2019s unique selling point and environmental benefits. The phrase \u201cChart a greener course for our planet\u201d is particularly effective, as it evokes a sense of adventure and responsibility. Well done.<\/p>\n<p><strong>Copywriter:<\/strong>\u00a0Thank you. I\u2019m glad you found the revised copy effective. If there\u2019s anything else you need assistance with, feel free to ask.<\/p>\n<p><strong>Art Director:<\/strong>\u00a0You\u2019re welcome. Your revised copy is indeed effective. Keep up the good work. If you need further assistance, don\u2019t hesitate to ask.<\/p>\n<p>Using Dall-E2, the\u00a0<strong>Art Director<\/strong>\u00a0goes off and sketches up an illustration to accompany the copy:<\/p>\n<div class=\"table-responsive\">\n<table>\n<tbody>\n<tr>\n<td><a class=\"lightbox-link\" href=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2023\/11\/jmsnap2023-11-02-at-16.04.33@2x.png\" aria-label=\"Lightbox image, click or press enter to enlarge\" data-featherlight=\"image\"><img decoding=\"async\" class=\"size-full wp-image-1435 aligncenter\" src=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2023\/11\/jmsnap2023-11-02-at-16.04.33@2x.png\" sizes=\"(max-width: 1594px) 100vw, 1594px\" srcset=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2023\/11\/jmsnap2023-11-02-at-16.04.33@2x.png 1594w, https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2023\/11\/jmsnap2023-11-02-at-16.04.33@2x-300x265.png 300w, https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2023\/11\/jmsnap2023-11-02-at-16.04.33@2x-1024x903.png 1024w, https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2023\/11\/jmsnap2023-11-02-at-16.04.33@2x-768x677.png 768w, https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2023\/11\/jmsnap2023-11-02-at-16.04.33@2x-1536x1355.png 1536w\" alt=\"Image jmsnap2023 11 02 at 16 04 33 2x\" width=\"256\" \/><\/a><\/td>\n<td>\n<h3>Recycled Egg-Carton Maps: Chart a greener course for our planet, one carton at a time.<button class=\"linkicon\" title=\"\" aria-label=\"Copy Post URL\" data-id-href=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/?p=1428&amp;preview_id=1428&amp;preview_nonce=2755ad3571&amp;post_format=standard&amp;_thumbnail_id=1429&amp;preview=true#recycled-egg-carton-maps-chart-a-greener-course-for-our-planet-one-carton-at-a-time\" data-toggle=\"tooltip\" data-placement=\"right\" data-original-title=\"Copy Post URL\"><i class=\"fabric-icon fabric-icon--Link\"><\/i><\/button><\/h3>\n<div class=\"screenreader-text\" aria-atomic=\"true\" aria-live=\"polite\"><\/div>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/blockquote>\n<p>This is definitely destined for a Cannes Lion advertising industry award. Right?<\/p>\n<h2>More productive AutoGen agents With Semantic Kernel<button class=\"linkicon\" title=\"\" aria-label=\"Copy Post URL\" data-id-href=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/?p=1428&amp;preview_id=1428&amp;preview_nonce=2755ad3571&amp;post_format=standard&amp;_thumbnail_id=1429&amp;preview=true#more-productive-autogen-agents-with-semantic-kernel\" data-toggle=\"tooltip\" data-placement=\"right\" data-original-title=\"Copy Post URL\"><i class=\"fabric-icon fabric-icon--Link\"><\/i><\/button><\/h2>\n<div class=\"screenreader-text\" aria-atomic=\"true\" aria-live=\"polite\"><\/div>\n<p>AutoGen is a framework that enables the development of LLM applications using multiple agents that can converse with each other to solve tasks. AutoGen agents are customizable, conversable, and seamlessly allow human participation. They can operate in various modes that employ combinations of LLMs, human inputs, and tools.<\/p>\n<p>Developing and finding the right tools to give AutoGen agents can be challenging though. Since this is an area where Semantic Kernel excels at, we wanted to demonstrate how you could give tools (i.e., plugins) to AutoGen that have already been imported into Semantic Kernel. These plugins could be authored as a collection of\u00a0<a href=\"https:\/\/learn.microsoft.com\/en-us\/semantic-kernel\/ai-orchestration\/plugins\/native-functions\/using-the-skfunction-decorator?tabs=Csharp\" target=\"_blank\" rel=\"noopener\">native functions<\/a>\u00a0<em>or<\/em>\u00a0you could\u00a0<a href=\"https:\/\/learn.microsoft.com\/en-us\/semantic-kernel\/ai-orchestration\/plugins\/chatgpt-plugins#running-the-plugin-with-semantic-kernel\" target=\"_blank\" rel=\"noopener\">import an existing OpenAI plugin<\/a>.<\/p>\n<p><div  class=\"d-flex justify-content-center\"><a class=\"cta_button_link btn-primary mb-24\" href=\"https:\/\/github.com\/microsoft\/semantic-kernel\/blob\/sk-autogen\/python\/notebooks\/12-sk-autogen-agents.ipynb\" target=\"_blank\">AutoGen Meets Semantic Kernel<\/a><\/div><\/p>\n<p dir=\"auto\">In our sample, we started by first creating two Semantic Kernel Plugins:<\/p>\n<ol dir=\"auto\">\n<li>A Bing Search Plugin, used to retrieve from Bing URLs for a given topic.<\/li>\n<li>A WebPages Plugin, used to fetch a given URL and return the text found.<\/li>\n<\/ol>\n<p>We then show how you can use these two Semantic Kernel plugins within the world of AutoGen. By giving an AutoGen agent access to these plugins, the AutoGen agent can pull real-time information to answer user questions.\u00a0<em>And<\/em>, because they\u2019re the\u00a0<em>only<\/em>\u00a0plugins we gave it, we can ensure its working within the safe boundaries provided by the kernel.<\/p>\n<p>Afterwards, a user can interact with AutoGen and it will use the plugins to:<\/p>\n<ol>\n<li>Search for an answer using Bing,<\/li>\n<li>Open a web page to get further details,<\/li>\n<li>Summarize the results as an answer back to the user.<\/li>\n<\/ol>\n<p dir=\"auto\"><div  class=\"d-flex justify-content-center\"><a class=\"cta_button_link btn-primary mb-24\" href=\"https:\/\/github.com\/microsoft\/semantic-kernel\" target=\"_blank\">Semantic Kernel Is Open Source<\/a><\/div><\/p>\n<p dir=\"auto\">Please be sure to \u2b50\ufe0f (star) the GitHub repo!<\/p>\n<hr \/>\n<h2 dir=\"auto\">Interested in learning more about agents?<\/h2>\n<p>There&#8217;s a <a href=\"https:\/\/learn.microsoft.com\/en-us\/archive\/msdn-magazine\/2016\/july\/csharp-applying-ai-to-a-multi-agent-mini-basketball-game\">vintage piece<\/a> written back in 2016 for Microsoft Developer Network Magazine that describes a multi-agent approach to using AI and simulating a basketball game. If you&#8217;re looking for something more up to date, check out Lilian Weng&#8217;s <a href=\"https:\/\/lilianweng.github.io\/posts\/2023-06-23-agent\/\">piece on autonomous agents<\/a> to get a sense of where this field is heading today.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this blog post, we show you how you can use Semantic Kernel with\u00a0AutoGen, a Microsoft Research project that shows the potential of using multiple agents together. With AutoGen, Microsoft research has shown that multiple agents can be better than one. To actually\u00a0do\u00a0work, however, agents within AutoGen need hooks into the real world so they [&hellip;]<\/p>\n","protected":false},"author":111267,"featured_media":1454,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[27,1],"tags":[],"class_list":["post-1428","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-agents","category-semantic-kernel"],"acf":[],"blog_post_summary":"<p>In this blog post, we show you how you can use Semantic Kernel with\u00a0AutoGen, a Microsoft Research project that shows the potential of using multiple agents together. With AutoGen, Microsoft research has shown that multiple agents can be better than one. To actually\u00a0do\u00a0work, however, agents within AutoGen need hooks into the real world so they [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/1428","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/users\/111267"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/comments?post=1428"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/1428\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media\/1454"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media?parent=1428"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/categories?post=1428"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/tags?post=1428"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}