Semantic Kernel Plugins<\/h2>\n
Semantic Kernel’s plugin architecture provides a structured solution. Plugins act as standardized wrappers around agent capabilities, enabling the kernel (the central orchestrator) to discover, invoke, and manage them effectively. Key benefits include:<\/p>\n
- \n
- Modular Agent Integration<\/strong>: Encapsulate each agent’s logic within a dedicated plugin.<\/li>\n
- Declarative Invocation<\/strong>: Define agent capabilities with clear descriptions, allowing the kernel’s LLM to dynamically choose the appropriate tool(s) based on the user query and context.<\/li>\n
- Unified Interface<\/strong>: Standardize communication, simplifying interactions between the orchestrator and diverse agents.<\/li>\n
- Centralized Monitoring<\/strong>: Track resource usage (like tokens) and execution flow across all agents.<\/li>\n
- Simplified Result Aggregation<\/strong>: Provide hooks to combine outputs from multiple agents into a cohesive final response.<\/li>\n<\/ol>\n
Architecture Overview<\/h2>\n
Let’s examine a conceptual architecture for an orchestration system using SK plugins:<\/p>\n
![\"sk](\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2025\/05\/sk-multi-architect.jpg\")
<\/a><\/p>\nThis architecture features:<\/p>\n
- \n
- A central\u00a0Orchestrator Agent<\/strong>, powered by Semantic Kernel, receiving user queries.<\/li>\n
- Multiple\u00a0Specialized Agent Plugins<\/strong>, each exposing the capabilities of a specific agent (e.g., Web Search, Org Policy Lookup, Private Data Query).<\/li>\n
- A\u00a0Plugin Registration<\/strong>\u00a0mechanism making agent capabilities discoverable by the kernel.<\/li>\n
- Result Aggregation<\/strong>\u00a0logic within the orchestrator to synthesize a final answer. The kernel uses the descriptions provided within each plugin (via decorators like\u00a0
@kernel_function<\/code>) to determine which plugin(s) are relevant to the user’s query, enabling dynamic and context-aware agent invocation.<\/li>\n<\/ul>\nImplementation Example<\/h2>\n
Consider this simplified Python implementation of an orchestrator agent using the Semantic Kernel SDK:<\/p>\n
class<\/span> OrchestratorAgent<\/span>:\r\n async<\/span> def<\/span> invoke<\/span>(self, message: ChatMessage, context: Context<\/span>) -> ChatResponseMessage:\r\n # 1. Set up the system prompt guiding the LLM orchestrator<\/span>\r\n self._history.add_system_message(self.system_prompt)\r\n self._history.add_user_message(message.user_query)\r\n\r\n # 2. Initialize specialized agent plugins<\/span>\r\n org_policy_agent_plugin = OrgPolicyAgentInvokingPlugin(kernel=self._kernel, message=message)\r\n web_search_agent_plugin = WebSearchAgentInvokingPlugin(kernel=self._kernel, message=message)\r\n private_data_agent_plugin = PrivateDataAgentInvokingPlugin(kernel=self._kernel, message=message)\r\n generic_agent_plugin = GenericAgentInvokingPlugin(kernel=self._kernel, message=message) # Fallback agent<\/span>\r\n\r\n # 3. Register plugins with the kernel, providing descriptive names<\/span>\r\n self._kernel.add_plugin(org_policy_agent_plugin, plugin_name=\"ORG_POLICY_AGENT\"<\/span>)\r\n self._kernel.add_plugin(web_search_agent_plugin, plugin_name=\"WEB_SEARCH_AGENT\"<\/span>)\r\n self._kernel.add_plugin(private_data_agent_plugin, plugin_name=\"PRIVATE_DATA_AGENT\"<\/span>)\r\n self._kernel.add_plugin(generic_agent_plugin, plugin_name=\"GENERIC_LLM_AGENT\"<\/span>)\r\n\r\n # Configure the kernel to automatically invoke functions based on the prompt and plugin descriptions<\/span>\r\n # execution_settings = ... # Configure execution settings, e.g., FunctionChoiceBehavior.Auto()<\/span>\r\n\r\n # 4. Invoke the kernel. The underlying LLM uses the prompt and function descriptions<\/span>\r\n # to decide which plugin functions to call (sequentially or in parallel).<\/span>\r\n results = []\r\n # Assuming self._sk_agent uses the configured kernel and execution settings<\/span>\r\n async<\/span> for<\/span> content in<\/span> self._sk_agent.invoke(self._history): # Potentially involves multiple plugin calls<\/span>\r\n results.append(content)\r\n\r\n # 5. Aggregate results from all potentially invoked plugins<\/span>\r\n # (Plugins should track their own invocation status and results)<\/span>\r\n agent_invoking_plugins = [\r\n org_policy_agent_plugin,\r\n web_search_agent_plugin,\r\n private_data_agent_plugin,\r\n generic_agent_plugin\r\n ] # List of all potential plugins<\/span>\r\n\r\n # 6. Calculate total token usage across invoked plugins<\/span>\r\n # (Requires plugins to expose usage data)<\/span>\r\n total_prompt_tokens = sum<\/span>(plugin.token_usage.prompt_token for<\/span> plugin in<\/span> agent_invoking_plugins if<\/span> plugin.was_invoked) # Example: Check if invoked<\/span>\r\n total_completion_tokens = sum<\/span>(plugin.token_usage.completion_token for<\/span> plugin in<\/span> agent_invoking_plugins if<\/span> plugin.was_invoked)\r\n\r\n # 7. Compile search results\/citations from invoked plugins<\/span>\r\n # (Requires plugins to expose relevant results)<\/span>\r\n search_result = SearchResult(docs=[], citations=[])\r\n for<\/span> plugin in<\/span> agent_invoking_plugins:\r\n if<\/span> plugin.was_invoked: # Example: Check if invoked<\/span>\r\n search_result.docs.extend(plugin.search_result.docs)\r\n search_result.citations.extend(plugin.search_result.citations)\r\n\r\n # 8. Return the consolidated response (likely the final LLM synthesis from step 4)<\/span>\r\n final_content = results[-1<\/span>].content if<\/span> results else<\/span> \"Could not generate a response.\"<\/span>\r\n return<\/span> ChatResponseMessage(\r\n content=final_content,\r\n search_result=search_result,\r\n token_usage=TokenUsage(\r\n prompt_token=total_prompt_tokens,\r\n completion_token=total_completion_tokens,\r\n ),\r\n )\r\n\r\n# Note: Requires AgentInvokingPlugin base class to define\/manage<\/span>\r\n# token_usage, search_result, was_invoked attributes, etc.<\/span>\r\n<\/code><\/pre>\nThis example highlights how plugins are registered and how the kernel orchestrates their execution, followed by result aggregation. (Note: Error handling and detailed state management are omitted for brevity).<\/p>\n
Anatomy of an Agent Plugin: WebSearchAgentInvokingPlugin<\/h2>\n
Let’s examine the\u00a0
WebSearchAgentInvokingPlugin<\/code>\u00a0more closely.<\/p>\n# Assumes AgentInvokingPlugin base class handles kernel, message, runtime, etc.<\/span>\r\nclass<\/span> WebSearchAgentInvokingPlugin<\/span>(AgentInvokingPlugin<\/span>):\r\n @kernel_function(\r\n name=\"invoke_web_search_agent\"<\/span>,\r\n description=\"\"\"WebSearch plugin. Use this to answer questions requiring current, public information, such as:\r\n - General knowledge questions or facts (e.g., 'What is the capital of France?')\r\n - Current events, market data, or industry trends (e.g., 'Latest stock price for MSFT?')\r\n - Technical information likely found online (e.g., 'How to configure a Flask server?')\r\n - Information published in academic papers or public industry reports.\"\"\"<\/span>,\r\n <\/span>)<\/span>\r\n async<\/span> def<\/span> invoke_web_search_agent<\/span>(\r\n self,\r\n query: Annotated[str<\/span>, \"The specific question or topic to search the web for. Should be self-contained.\"<\/span>],\r\n <\/span>):\r\n \"\"\"Invokes the external Web Search Agent.\"\"\"<\/span>\r\n # self.was_invoked = True # Mark as invoked<\/span>\r\n message_body = self.create_message_body(query) # Prepare agent-specific message<\/span>\r\n recipient = AgentId(WEBSEARCH_AGENT, str<\/span>(self.message.metadata.session_id)) # Target agent<\/span>\r\n\r\n # Send message via runtime and await response<\/span>\r\n result = await<\/span> self.runtime.send_message(message=message_body, recipient=recipient)\r\n\r\n # Process result, update internal state (token usage, search results), and return<\/span>\r\n processed_response = self.proccess_and_create_response(result)\r\n # self.token_usage = ... # Update token usage based on result<\/span>\r\n # self.search_result = ... # Update search results based on result<\/span>\r\n return<\/span> processed_response # Return processed data, often just a summary string for the orchestrator LLM<\/span>\r\n<\/code><\/pre>\nKey aspects enabling orchestration:<\/p>\n
1. Clear Domain Boundaries via\u00a0
@kernel_function<\/code><\/h3>\nThe\u00a0
description<\/code>\u00a0parameter in\u00a0@kernel_function<\/code>\u00a0is crucial. The orchestrator LLM uses this natural language description to understand\u00a0when<\/em>\u00a0to use this specific plugin. A well-crafted description significantly improves the accuracy of dynamic function calling.<\/p>\n2. Informative Parameter Signatures with\u00a0
Annotated<\/code><\/h3>\nType hints (
str<\/code>) and\u00a0Annotated<\/code>\u00a0descriptions (\"The specific question...\"<\/code>) provide further context to the LLM, helping it formulate the correct input (query<\/code>) for the function based on the overall user request.<\/p>\n3. Encapsulated Agent Communication Logic<\/h3>\n
The plugin abstracts the details of interacting with the specific agent (creating messages, identifying recipients, handling runtime communication). The orchestrator only needs to know about the high-level\u00a0
invoke_web_search_agent<\/code>\u00a0function.<\/p>\n4. Standardized Result Processing<\/h3>\n
The\u00a0
proccess_and_create_response<\/code>\u00a0method (part of the base class or implemented here) normalizes the agent’s raw output into a format the orchestrator expects. It also updates shared state like token counts and extracted citations, essential for the final aggregation step.<\/p>\nHierarchical Plugin Structure<\/h2>\n
Semantic Kernel supports organizing plugins hierarchically, which can mirror complex agent interactions:<\/p>\n
![\"sk](\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2025\/05\/sk-multi-architect-hierarchy.jpg\")
<\/a><\/p>\nAn\u00a0
AgentInvokingPlugin<\/code>\u00a0might internally use other, more granular plugins (e.g., a\u00a0BingSearchPlugin<\/code>\u00a0or an\u00a0HttpRequestPlugin<\/code>). This allows for composition and reuse of lower-level functionalities within the agent abstraction.<\/p>\nThe Power of Plugin-Based Orchestration<\/h2>\n
This SK plugin approach yields significant advantages:<\/p>\n
1. Dynamic & Context-Aware Agent Selection<\/h3>\n
The kernel’s LLM acts as the intelligent router, using function descriptions and conversation history to select the most relevant agent(s) for each turn or sub-task. The\u00a0
system_prompt<\/code>\u00a0guides this selection process:<\/p>\n# Example system prompt snippet for the orchestrator LLM<\/span>\r\nsystem_prompt = \"\"\"You are an AI assistant orchestrating specialized agents. Analyze the user query.\r\nBased on the query and the descriptions of available tools (plugins), select the most appropriate tool(s).\r\nAvailable tools:\r\n- ORG_POLICY_AGENT: Use for questions about internal company policies and procedures.\r\n- WEB_SEARCH_AGENT: Use for current events, public facts, and general web knowledge.\r\n- PRIVATE_DATA_AGENT: Use for queries requiring access to internal databases (e.g., sales figures, user data).\r\n- GENERIC_LLM_AGENT: Use as a fallback for general conversation or if no other tool is suitable.\r\nInvoke the chosen tool(s). If multiple tools are needed, plan their execution. Synthesize their outputs into a final, comprehensive response.\r\n\"\"\"<\/span>\r\n<\/code><\/pre>\n2. Flexible Execution Flows (Parallel\/Sequential)<\/h3>\n
SK supports different function calling modes. You can configure the kernel to:<\/p>\n
- \n
- Invoke multiple plugins in parallel if their tasks are independent.<\/li>\n
- Chain plugin calls sequentially if one agent’s output is needed as input for another.<\/li>\n
- Allow the LLM to automatically decide the best execution strategy.<\/li>\n<\/ul>\n
3. Unified Resource Monitoring<\/h3>\n
By having plugins report their token usage and execution time back to the orchestrator, you gain centralized visibility into costs and performance bottlenecks across the entire multi-agent system.<\/p>\n
Implementation Patterns<\/h2>\n
1. Base Class for Agent Invocation Plugins<\/h3>\n
Create a base class to handle common logic like kernel\/message storage, runtime access, state tracking (invocation status, token usage, results), and result processing.<\/p>\n
from<\/span> semantic_kernel.kernel_pugin import<\/span> KernelPlugin\r\n# ... other imports ...<\/span>\r\n\r\nclass<\/span> AgentInvokingPlugin<\/span>(KernelPlugin<\/span>):\r\n def<\/span> __init__<\/span>(self, kernel, message, runtime<\/span>):\r\n self.kernel = kernel\r\n self.message = message\r\n self.runtime = runtime # Agent communication runtime<\/span>\r\n self.token_usage = TokenUsage(prompt_token=0<\/span>, completion_token=0<\/span>)\r\n self.search_result = SearchResult(docs=[], citations=[])\r\n self.was_invoked = False<\/span>\r\n # Potentially add abstract methods for processing results, etc.<\/span>\r\n\r\n def<\/span> create_message_body<\/span>(self, query: str<\/span><\/span>) -> dict<\/span>:\r\n # Standardize message creation<\/span>\r\n pass<\/span>\r\n\r\n def<\/span> proccess_and_create_response<\/span>(self, agent_result: dict<\/span><\/span>) -> str<\/span>:\r\n # Standardize response processing, update state (tokens, results)<\/span>\r\n # Returns a string summary suitable for the orchestrator LLM<\/span>\r\n pass<\/span>\r\n\r\n def<\/span> get_summary<\/span>(self<\/span>) -> str<\/span>:\r\n # Return a summary of results if invoked<\/span>\r\n pass<\/span>\r\n\r\n # ... other common methods ...<\/span>\r\n<\/code><\/pre>\n2. Descriptive Plugin and Function Naming<\/h3>\n
Use clear, descriptive names for plugin registration (
plugin_name=\"WEB_SEARCH_AGENT\"<\/code>) and kernel functions (@kernel_function(name=\"invoke_web_search_agent\", ...)<\/code>). This aids both the LLM’s understanding and human debugging.<\/p>\n3. Explicit Function Choice Configuration<\/h3>\n
Leverage\u00a0
FunctionChoiceBehavior<\/code>\u00a0(or similar mechanisms in the specific SK SDK version) to control how the LLM selects and executes functions (e.g.,\u00a0Auto<\/code>\u00a0for dynamic selection,\u00a0Required<\/code>\u00a0to force a specific function call).<\/p>\nBest Practices for Agent Orchestration with SK Plugins<\/h2>\n
- \n
- Rich Function Descriptions<\/strong>: Write detailed, unambiguous descriptions for\u00a0
@kernel_function<\/code>. This is paramount for accurate dynamic routing.<\/li>\n- Clear System Prompts<\/strong>: Guide the orchestrator LLM on the overall goal, available tools, and how to choose between them.<\/li>\n
- Standardized Inputs\/Outputs<\/strong>: Design plugin functions to accept simple inputs (like strings) where possible and return standardized, predictable outputs (or update state predictably).<\/li>\n
- Robust Error Handling<\/strong>: Implement try\/except blocks within plugin functions and report errors clearly back to the orchestrator.<\/li>\n
- Comprehensive Telemetry<\/strong>: Log key events (plugin invocation, agent requests\/responses, errors, timings) for monitoring and debugging. Use correlation IDs.<\/li>\n
- State Management<\/strong>: Carefully manage shared state (like conversation history) passed to plugins and how plugins update aggregated results (tokens, citations).<\/li>\n
- Observability First<\/strong>: Design plugins with logging and metrics in mind from the outset. Track token usage per plugin\/agent.<\/li>\n
- Agent Contract Testing<\/strong>: Implement tests for each plugin, mocking the underlying agent, to ensure it adheres to its expected interface and behavior.<\/li>\n
- Idempotency Considerations<\/strong>: If agents might be retried, consider designing their actions to be idempotent where possible.<\/li>\n
- Circuit Breakers<\/strong>: Implement resilience patterns like circuit breakers for calls to external agents that might be unreliable or slow.<\/li>\n<\/ol>\n
Conclusion<\/h2>\n
Semantic Kernel plugins offer a powerful and structured approach to the complex challenge of AI agent orchestration. By encapsulating agent capabilities within a standardized plugin framework, developers can build sophisticated multi-agent systems that benefit from:<\/p>\n
- \n
- Modularity<\/strong>: Easier development, testing, and maintenance.<\/li>\n
- Dynamic Routing<\/strong>: Intelligent, context-aware selection of specialized agents.<\/li>\n
- Scalability<\/strong>: A clear pattern for adding new agent capabilities.<\/li>\n
- Observability<\/strong>: Centralized monitoring of performance and resource consumption.<\/li>\n<\/ul>\n
As AI applications increasingly rely on collaborative ensembles of specialized agents, mastering plugin-based orchestration frameworks like Semantic Kernel will be essential for building the next generation of intelligent, capable, and maintainable systems.<\/p>\n
References:<\/h2>\n
- \n
- Semantic Kernel<\/a><\/li>\n
- Semantic Kernel Plugin<\/a><\/li>\n
- Function calling<\/a><\/li>\n<\/ul>\n
\nThis blog post targets software engineers familiar with LLM concepts and agent-based architectures. Code examples are illustrative and simplified; production systems require more extensive error handling, configuration, and state management.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"
Today we’re excited to welcome Jarre Nejatyab as a guest blog to highlight a technical deep dive on orchestrating AI Agents with Semantic Kernel Plugins. In the rapidly evolving world of Large Language Models (LLMs), orchestrating specialized AI agents has become crucial for building sophisticated cognitive architectures capable of complex reasoning and task execution. While […]<\/p>\n","protected":false},"author":149071,"featured_media":4764,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[117,1],"tags":[48,63,9],"class_list":["post-4751","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-guest-blog","category-semantic-kernel","tag-ai","tag-microsoft-semantic-kernel","tag-semantic-kernel"],"acf":[],"blog_post_summary":"
Today we’re excited to welcome Jarre Nejatyab as a guest blog to highlight a technical deep dive on orchestrating AI Agents with Semantic Kernel Plugins. In the rapidly evolving world of Large Language Models (LLMs), orchestrating specialized AI agents has become crucial for building sophisticated cognitive architectures capable of complex reasoning and task execution. While […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/4751","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/users\/149071"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/comments?post=4751"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/4751\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media\/4764"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media?parent=4751"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/categories?post=4751"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/tags?post=4751"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
- Semantic Kernel Plugin<\/a><\/li>\n
- Dynamic Routing<\/strong>: Intelligent, context-aware selection of specialized agents.<\/li>\n
- Clear System Prompts<\/strong>: Guide the orchestrator LLM on the overall goal, available tools, and how to choose between them.<\/li>\n
- Rich Function Descriptions<\/strong>: Write detailed, unambiguous descriptions for\u00a0
- Multiple\u00a0Specialized Agent Plugins<\/strong>, each exposing the capabilities of a specific agent (e.g., Web Search, Org Policy Lookup, Private Data Query).<\/li>\n
- Declarative Invocation<\/strong>: Define agent capabilities with clear descriptions, allowing the kernel’s LLM to dynamically choose the appropriate tool(s) based on the user query and context.<\/li>\n