{"id":4005,"date":"2025-01-15T16:32:59","date_gmt":"2025-01-16T00:32:59","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/semantic-kernel\/?p=4005"},"modified":"2025-02-10T10:17:09","modified_gmt":"2025-02-10T18:17:09","slug":"guest-blog-building-multi-agent-systems-with-multi-models-in-semantic-kernel-part-1","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/agent-framework\/guest-blog-building-multi-agent-systems-with-multi-models-in-semantic-kernel-part-1\/","title":{"rendered":"Guest Blog: Building Multi-Agent Systems with Multi-Models in Semantic Kernel \u2013 Part 1"},"content":{"rendered":"

Today I want to welcome back Arafat Tehsin, who\u2019s a Microsoft Most Valuable Professional (MVP) for AI. He\u2019s written an article we\u2019re sharing below, focused on Building Multi-Agent Systems with Multi-Models in Semantic Kernel<\/a>. We\u2019ll turn it over to Arafat to share more!<\/p>\n

\"practical<\/p>\n

Imagine running a restaurant. Each of your team member, from the chef to server to the cashier has got specialised skills. While the chef prepares delicious meals, the server ensures that the food is delivered promptly and the cashier ensures smooth transactions. Together, they deliver a seamless dining experience.<\/p>\n

Similarly, in a multi-agent system powered by\u00a0Semantic Kernel<\/a>, each AI agent specialises in its task, such as generating travel plans, analysing budget or creating marketing campaign banners. These agents communicate and collaborate seamlessly, like a well-coordinated restaurant team, to tackle complex, multifaceted problems efficiently to achieve overarching goals.<\/p>\n

Why do we need Agents?<\/h3>\n

Agents are autonomous or semi-autonomous software entity designed to perform tasks by processing inputs, making decisions and producing outputs. They introduce modularity and adaptability to AI systems. They provide capabilities like collaboration, autonomy and human-agent interactions. For example, in a customer support system, one agent might analyse user queries while other fetches data from databases and a third one generates a response.<\/p>\n

Why Multi-Agent Systems?<\/h3>\n

While a single agent can tackle specific tasks effectively (see below<\/em>), complex problems often demand collaboration. Multi-agent systems introduce distributed intelligence, allowing agents to specialise and coordinate tasks in parallel. For example, in a dynamic sales process, one agent could monitor social media trends, another analyses customer behavior and a third provides actionable insights. This way, they work together for a holistic strategy.<\/p>\n

Why Multi-Model Systems?<\/h3>\n

Integrating multiple models into such systems amplifies their versatility. A single agent or even a group of agents tied to one model can be limiting, particularly when diverse tasks demand specialised capabilities. By using multi-model orchestration, agents to seamlessly use different models to get the most out of frontier models. For example, GPT-4 handles customer interactions, Gemini Ultra interprets visual data from a damaged product and Llama 3.2 ensures real-time translations for global communication.<\/p>\n

Integration of all of the above, enterprises can achieve faster operational efficiency and adaptability. These systems can enable businesses to harness AI\u2019s full potential by delivering personalised customer experiences with optimal innovation.<\/p>\n

Let\u2019s talk about building these agents<\/h3>\n

Semantic Kernel provides an\u00a0Agent Framework<\/a>\u00a0to build these agents. There are certain basic concepts which I just summarise here as I do not want to just rehash the Microsoft Learn docs here.<\/p>\n

This framework is an experimental state. Its architecture designed to enable the creation and management of intelligent agents. It allows multiple agents to collaborate within a single conversation and manage multiple concurrent conversations including the multi-model and multimodal capabilities. The framework includes core components like the\u00a0Agent<\/code>\u00a0class,\u00a0Agent Chat<\/code>\u00a0and\u00a0Agent Channel<\/code>, which facilitate agent interactions and collaboration.<\/p>\n

It integrates with the Semantic Kernel\u2019s core principles by ensuring consistency, enabling advanced and autonomous agent behaviors. Additionally, plugins and function calling enables the agents\u2019 capabilities by allowing them to interact with external services and execute complex tasks.<\/p>\n

This time, I took a different approach to guide you as a developer on how you can learn all of the Agent Framework\u2019s capabilities swiftly. As you may remember, I have created a\u00a0getting started notebook for Semantic Kernel<\/a>\u00a0which you can learn step by step by following the examples. I won\u2019t write about the details which are already included in the latest\u00a0Agents<\/code>\u00a0notebook. However, I thought to give you a brief overview of agents\u00a0I have created<\/a>\u00a0as a part of notebook.<\/p>\n

Kitchen Assistant (Single Agent)<\/h3>\n

A single agent can be considered as a smart assistant that is specialised in a certain area. Considering below diagram, it illustrates how\u00a0Food Agent<\/strong>\u00a0functions within a smart kitchen assistant to provide personalised meal recommendations. The process begins with the user\u2019s query, such as\u00a0\u201cWhat should I eat right now?\u201d<\/em>, routed through\u00a0InvokeAgentAsync<\/code>\u00a0<\/b>to the\u00a0Food Agent<\/strong>. Powered by the\u00a0foodAgentKernel<\/code>, the agent uses integrated plugins like the\u00a0TimePlugin<\/code>\u00a0(to factor in the time of day) and\u00a0FoodPlugin<\/code>\u00a0<\/b>(to access recipes and ingredients) to generate a tailored meal suggestion. It also leverages\u00a0ChatHistory<\/code>\u00a0to maintain context, ensuring a seamless and personalized interaction.<\/p>\n

The\u00a0foodAgent<\/code>\u00a0creates meal recommendations that consider the weather, time of day, and dietary preferences (e.g., Halal or vegetarian). It provides recipes alongside its suggestions, making the experience user-friendly and interactive. This overall system showcases how a single, well-coordinated agent can dynamically adapt to user needs by delivering thoughtful and relevant results with minimal input.<\/p>\n

\"Single<\/p>\n

As you may notice at the start of the notebook, we\u2019re aiming to use Azure OpenAI, Google Gemini and Meta\u2019s Llama to utilise multi-model capabilities for better reasoning and outputs. It also doesn\u2019t take a lot of effort to switch the models as well. For example, if you see that I am using GPT-4 for weather service, you can always change it to Google Gemini to see experience it differently.<\/p>\n

Here\u2019s the output it generated:<\/p>\n

\"Food<\/p>\n

Concierge Service (Multi Agents)<\/h3>\n

The following diagram showcases the workflow of an AI Concierge Service coordinated by an\u00a0Agent Group Chat<\/strong>. The process begins with the\u00a0User<\/strong>\u00a0submitting a request for a travel plan. The\u00a0Agent Group Chat<\/strong>\u00a0directs the task to the\u00a0Travel Planner Agent<\/strong>, which creates an itinerary and calculates the budget. This itinerary and budget are then sent to the\u00a0Budget Advisor Agent<\/strong>, which checks the feasibility of the proposed plan within the specified budget.<\/p>\n

\"\"<\/p>\n

The\u00a0Budget Advisor Agent<\/strong>\u00a0either approves the plan (\u2705) or rejects it (\ud83d\udeab). This decision is relayed back to the\u00a0Agent Group Chat<\/strong>, which communicates the final outcome to the\u00a0User<\/strong>. The Semantic Kernel Agent Framework manages collaboration between specialised agents, delivering a well-considered travel plan while adhering to financial constraints. It highlights the seamless coordination among agents to meet user requirements. There\u2019s a lot to it which you can surely explore within the\u00a0agents-01<\/strong>\u00a0notebook.<\/p>\n

\n
\n
\n
\n
\"slider<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n

.NET has received an AI boost in the shape of Semantic Kernel, once again.\u00a0Building multi-agent systems with multi-models<\/a>\u00a0have never been unified and simpler than ever before with this awesome framework. Not just with Microsoft or OpenAI offerings, you can also use all the other frontier models to build productive agents and enhance their overall performances. This post covers (including my Generative AI for Developers\u00a0notebook<\/a>) everything that is good for a beginner or junior developer stepping into the AI ecosystem.<\/p>\n

However, there\u2019s one thing which I did not talk about\u2026 and that\u2019s\u00a0human in the loop interactions.\u00a0<\/em>I\u2019m aiming it to cover in our next post with some advanced features. Until next time.<\/p>\n

Thanks Arafat from the Semantic Kernel team, we appreciate you sharing your work and look forward to part 2 on human in the loop interactions. Please reach out if you have any questions or feedback through our\u00a0Semantic Kernel GitHub Discussion Channel<\/a>. We look forward to hearing from you!<\/p>\n","protected":false},"excerpt":{"rendered":"

Today I want to welcome back Arafat Tehsin, who\u2019s a Microsoft Most Valuable Professional (MVP) for AI. He\u2019s written an article we\u2019re sharing below, focused on Building Multi-Agent Systems with Multi-Models in Semantic Kernel. We\u2019ll turn it over to Arafat to share more! Imagine running a restaurant. Each of your team member, from the chef […]<\/p>\n","protected":false},"author":149071,"featured_media":2302,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[117],"tags":[48,14,63,9],"class_list":["post-4005","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-guest-blog","tag-ai","tag-azureopenai","tag-microsoft-semantic-kernel","tag-semantic-kernel"],"acf":[],"blog_post_summary":"

Today I want to welcome back Arafat Tehsin, who\u2019s a Microsoft Most Valuable Professional (MVP) for AI. He\u2019s written an article we\u2019re sharing below, focused on Building Multi-Agent Systems with Multi-Models in Semantic Kernel. We\u2019ll turn it over to Arafat to share more! Imagine running a restaurant. Each of your team member, from the chef […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/4005","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/users\/149071"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/comments?post=4005"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/4005\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media\/2302"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media?parent=4005"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/categories?post=4005"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/tags?post=4005"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}