The Semantic Kernel Agent Framework revolutionizes how developers can interact with Large Language Models (LLMs) by embedding dynamic, multi-step agents into their applications. By combining the power of LLMs with structured programming, the framework allows developers to build intelligent systems that can autonomously carry out tasks, reason based on context, and collaborate across multiple agents.

In this blog, we’ll dive into the Agent Framework documentation now available — exploring its architecture, key components, and step-by-step guides on implementing different agents using the SDK.

What is the Agent Framework?

The Agent Framework within Semantic Kernel allows developers to create and configure agents that autonomously carry out tasks based on user input. These agents leverage LLMs to process natural language commands, combine them with pre-defined programming logic, and execute complex actions. Whether it’s completing a chat, running calculations, or searching files, agents act as intelligent intermediaries between user requests and programmatic execution.

Architecture: How Do Agents Operate?

The Agent Framework Architecture is built upon the principles of Semantic Kernel, providing a robust foundation for implementing sophisticated agent functionalities. It enables multiple agents to collaborate within a single conversation, incorporating human input, and allows for managing multiple concurrent discussions.

The framework benefits from the abstract Agent class, which serves as a blueprint for all agents, including specialized types like the Chat Completion Agent and the Open AI Assistant Agent. By leveraging the Kernel’s capabilities, agents can perform tasks autonomously or interact dynamically within an Agent Chat. This flexible arrangement supports dynamic multi-agent systems that are adaptable to various conversational and task-driven scenarios, enhancing applications such as customer support and collaborative problem-solving. Developers can integrate custom functionalities through Plugins and harness Function Calling, expanding agent capabilities and ensuring seamless adaptation from traditional chat applications to advanced agent-driven systems.

Chat Completion Agent

The Chat Completion Agent extends the familiar capabilities of Semantic Kernel’s Chat Completion Service, offering developers a seamless way to integrate sophisticated conversational AI into their applications. Leveraging the protocol of maintaining chat-history with AI models, this agent acts as an intelligent chatbot that can understand and respond to user inputs while maintaining context across multiple exchanges. By building on Semantic Kernel’s existing framework, the Chat Completion Agent can utilize any AI service supported by the kernel, selecting the appropriate model through a service selector for tailored interactions. This makes it particularly effective for developing applications like customer support bots and virtual assistants, where multi-turn dialogue and task completion are critical. Through the dynamic generation of responses powered by LLMs, the agent enhances its interactions to meet evolving user needs, offering a powerful tool for any application relying on natural language interaction.

OpenAI Assistant Agent

Building on the powerful V2 APIs of OpenAI’s assistant models, the OpenAI Assistant Agent provides a sophisticated interface for bridging LLM capabilities with programmatic actions. Unlike the Chat Completion Agent, which relies on a client-side chat history to manage conversations, the OpenAI Assistant Agent utilizes a server-side thread to maintain and process dialogue. Conversations are managed as threads where Semantic Kernel messages are seamlessly integrated and processed. This server-side architecture enhances the agent’s ability to generate text, answer questions, and trigger system actions such as retrieving files or interacting with APIs. Ideal for applications like virtual assistants or automated research tools, the agent enables both processing and execution of tasks in tandem. By leveraging OpenAI’s advanced API and a robust threading model, it ensures contextual continuity and dynamic execution of actions, thus improving both conversational and task-driven applications.

Agent Collaboration

In more complex systems, multiple agents may need to collaborate to achieve a common goal. The Agent Framework supports Agent Collaboration, allowing multiple agents to interact and share data in real time. These agents can split tasks, communicate results to each other, and work in parallel to solve larger, multi-faceted problems.

For example, one agent might gather customer data while another processes this data and sends follow-up emails. This distributed task handling enables more efficient workflows and enhances the overall performance of your applications.

Create an Agent from a Template

Building an agent from scratch can be complex, but Semantic Kernel streamlines this process through versatile prompting capabilities. Users can define YAML prompts to establish the agent, its execution settings, and allowed function choice behaviors, providing a structured foundation for development. Additionally, Semantic Kernel supports other prompt templates like Handlebars—available in both C# and Python—or Jinja2 for Python, enabling developers to choose the format that best suits their needs.

These prompt templates serve as a starting point for quickly launching agents tailored to common scenarios such as chatbots, file search, or code interpretation. This templated approach not only accelerates development by pre-defining essential skills and actions but also allows for extensive customization to meet the specific demands of your application. By focusing on refining agent behaviors rather than foundational setup, you can ensure a seamless integration and operation tailored to your unique requirements.

Note: Defining an Agent via a Prompt Template is coming soon to Python.

Configuring Agents with Plugins

Plugins allow you to extend the capabilities of your agents without starting from scratch. The Plugins system in the Agent Framework allows developers to add new features, such as external data connectors, custom logic, or integration with third-party APIs, with minimal effort.

For instance, you can add a CRM connector plugin to your chatbot, enabling it to pull customer data in real-time during a conversation. Plugins make agents more versatile, allowing them to carry out more complex actions and integrate more deeply into business processes.

Streaming Agent Responses

For real-time applications, the ability to deliver results incrementally is crucial. The Agent Framework supports streaming responses, allowing agents to provide feedback in real time as they work through a task. This ensures that users receive immediate updates or partial results, which enhances the overall experience, especially in time-sensitive applications like data analysis or customer support.

How-To: Implementing Different Agents

Below are step-by-step guides for building and configuring various agents in Semantic Kernel:

Chat Completion Agent

In this walkthrough, we will configure a Chat Completion Agent to work with the GitHub API, leveraging Semantic Kernel’s capabilities for an enriched task execution. We’ll define the dialogue type and implement skills necessary for the agent to interpret and generate responses effectively. This process includes configuring natural language understanding, intent recognition, and managing multi-turn conversations.

Specifically, the agent will utilize a plugin to interact with the GitHub API, enabling it to answer queries about GitHub repositories with templatized instructions. Each response will include document citations for clarity and reference. By employing streaming, the agent provides real-time updates, allowing users to see the progress of their queries dynamically. This step-by-step guide will highlight the critical components of coding, making it easy to replicate and adapt for various use cases.

C# Example

Python Example

Assistant Agent – Code Interpreter

In this guide, we’ll cover using the code-interpreter functionality of an Open AI Assistant Agent to execute data-analysis tasks. This involves step-by-step insights into the coding process, emphasizing how the agent can parse and analyze code snippets, offering feedback or corrections. This capability is vital for developer tools, where the agent can aid in debugging or enhancing code quality.

Beyond corrections, the code interpreter, in conjunction with LLMs, can simplify and explain complex code sections to users, making it an invaluable educational tool. As part of the data-analysis task, the agent will generate both image and text outputs, showcasing the tool’s versatility in performing quantitative analysis. Streaming will be employed to deliver the agent’s responses, ensuring that users receive real-time updates as the analysis progresses, enhancing interaction and productivity.

C# Example

Python Example

Assistant Agent – File Search

In this tutorial, we will explore utilizing the file-search capability of an Open AI Assistant Agent to perform comprehension tasks. The process will be outlined step-by-step, ensuring clarity and precision, as the agent navigates and retrieves the necessary documents. By breaking down queries, the agent searches specific directories or databases, retrieving pertinent files and filtering results using parameters like file type, creation date, or content keywords.

Throughout the task, the agent will enhance responses by including document citations, thus providing a well-rounded comprehension output. Streaming will facilitate real-time updates, allowing users to receive immediate insights as the file search progresses. This approach highlights the agent’s ability to perform detailed searches efficiently, making it a powerful tool for comprehensive information retrieval tasks.

C# Example

Python Example

Agent Group Chat

In this example, we will show how to use an Agent Group Chat to orchestrate the collaboration between two agents tasked with reviewing and rewriting user-generated content. Each agent is assigned a specific role to optimize the workflow: the Reviewer evaluates the content and provides guidance, while the Writer updates the content according to the Reviewer’s feedback. This step-by-step guide will illuminate the key aspects of the coding process, demonstrating how to efficiently set up and manage such interactions.

For applications necessitating coordinated efforts from multiple agents, a Group Chat setup facilitates seamless communication and task distribution. One agent can manage user input, while others execute specialized tasks and report back, streamlining large-scale operations. This collaborative, multi-agent system is particularly beneficial in complex scenarios like customer support, where integrated systems must work in concert to deliver comprehensive solutions.

C# Example

Python Example

Wrapping Up

The Agent Framework is a powerful tool that empowers developers to build intelligent, multi-tasking agents capable of carrying out a wide variety of actions autonomously. Whether you’re building a chatbot, a virtual assistant, or a multi-agent collaboration system, the Agent Framework provides the structure and tools you need to succeed.

With its modular architecture, integration capabilities, and extensive documentation, the Agent Framework in Semantic Kernel is an essential resource for anyone looking to create next-generation AI-powered applications. Start exploring the Agent Framework documentation today to see how you can bring intelligent agents into your projects.

Please reach out if you have any questions or feedback through our Semantic Kernel GitHub Discussion Channel. We look forward to hearing from you! We would also love your support — if you’ve enjoyed using Semantic Kernel, give us a star on GitHub.