Building the next era of AI: Teams AI Library and API message extensions | Ignite 2023
Just as all software is expected to be connected by the internet, in a few years every software product will include an intelligent copilot assistant according to Kevin Scott, Microsoft’s chief technology officer. To help developers seize this opportunity, we released everything you need to build an intelligent copilot for your app on Teams. Today, we are simplifying the process of creating intelligent apps with the release of API message extensions and the Teams AI Library. With these tools, you can join multiple aspects of the AI era with ease from integrating your app into Microsoft’s Copilot and Teams in minutes with API message extensions to creating your own intelligent app with the Teams AI Library.
The fastest and easiest way to integrate your app with Teams and soon with Copilot is API message extensions. Available now, API message extensions enable users to search and share information from your app directly in the Teams client reducing the need to switch applications streamlining their workflows. Creating an API message extension with the Teams Toolkit is remarkably straightforward, all you need is an OpenAPI Description document to get started, and it can be built in minutes. As API message extensions will shortly be supported as Copilot plugins, using them enables your app to reach hundreds of millions of people across multiple products where they work every day.
To expand your app’s experience beyond Microsoft’s Copilot, the Teams AI Library brings your own AI-powered apps and copilots onto the Teams Platform! Available now with exciting updates, this powerful development toolkit allows developers to easily add AI components, such as conversational AI, advanced prompt management with augmentation, and safety moderation to create naturally engaging apps with reasoning and generative capabilities. Build your app’s own intelligent copilot and give users a customized AI experience interacting with them like never before.
Combined, this suite of developer tools offers users an end-to-end experience meeting them where they are, whether that be utilizing your app within the Copilot ecosystem or directly collaborating with your own intelligent AI assistant. Use both and dive into the next era of AI today!
The Teams AI Library: Integrate your copilot within Teams!
At Microsoft, our AI approach is to help you be successful in developing for an AI world. Building plugins for Copilot extends its value with your applications, exposed through Teams messages extensions or APIs. On the other hand, you can also tap into the power of a comprehensive pro-code development suite of tools in Azure AI Studio to customize and build AI powered experiences for your needs. Teams, with more than 320+ million monthly active users, offers a platform where you can take advantage of your own copilot to reach your users in their flow of work. Here is where the Teams AI Library plays a significant role to ease the integration of your AI powered solution within Teams.
With deeper integration in Teams through the AI Library, your app’s collaborative capabilities can be leveraged to their fullest extent. Conversational session history allows your AI assistant to remember context across Teams messages enabling natural conversations and relevant responses without requiring users to explicitly call out a registered set of actions. Your assistant can help users wherever they are collaborating, be it in chats, channels, or meetings. With access to the group roster, your app can seamlessly coordinate with others across the world. This opens new possibilities, such as updating members of a global team in any language on the latest product strategy or coordinating with each teammate on new project deliverables. Transform your user experience with an AI assistant working alongside users throughout their day on Teams.
Today, we’re excited to announce the Teams AI Library in general availability, a significant leap forward in our mission to empower developers like you in creating cutting-edge, AI-driven applications for Microsoft Teams. This refined toolkit is designed to simplify the development process, making it easier for you to construct intelligent, conversational app experiences that cater to your Teams users, serving as their very own copilots. With state-of-the-art AI and large language models at your disposal, you can assist users in tackling complex cognitive tasks, whether it’s crafting sales pitches, catching up on missed meetings, generating presentation images, or planning themed dinner parties. This improved Teams AI Library equips you with the tools needed to boost productivity and innovation, making your app’s experience on Teams more engaging. In the following section, we will delve into the key features and benefits of this enhanced library and demonstrate how it can revolutionize the way you build AI-powered Teams apps.
New feature highlights of the Teams AI Library
Bring OpenAI Assistants to Teams with Teams AI Library
In Preview, the AI Assistant Bot Template in the Teams Toolkit Brings OpenAI Assistants to Teams
Last week, OpenAI released Assistants API simplifying the way you build custom conversational AI bots on their platform. With new updates to the Teams Toolkit and the Teams AI Library, you can bring these assistants into your Teams applications in minutes.
An OpenAI Assistant leverages OpenAI models to respond to user queries and can be customized with simple instructions to incorporate different personalities and capabilities. The assistant can access files uploaded during their creation or during an interaction with users, making them a subject matter expert. Capable of accessing multiple tools in parallel, such as OpenAI’s Code Interpreter and Knowledge retrieval, your assistant can specialize in several roles whether it is serving as a human resources policy expert, a math teacher, or a personal tutor. Bringing the assistant into Teams allows you to layer on additional functionality with the Teams AI Library such as enhanced safety moderation and action execution while launching your app to hundreds of millions of Teams users.
Integrating your OpenAI Assistant with Teams is simple! After you create your OpenAI Assistant, use the Teams Toolkit Template AI Assistant Bot and the Teams AI Library to quickly create your Teams app in a structured and efficient way.
Steps to Bring your OpenAI Assistant to Teams:
- Install the pre-release version of Teams Toolkit: Install Teams Toolkit – Teams | Microsoft Learn
- Create a new Project: Create a new Teams app – Teams | Microsoft Learn
- In the project creation dialog, select Bot -> AI Assistant Bot
- Create an OpenAI Assistant
- Install the dependency packages
- Provide your OpenAI API Key and Assistant ID
- Launch your app to Teams
For more details, please refer to the README file available in the project after selecting the AI Assistant Bot template.
The Teams AI Library is built with flexibility and modularity in mind. Whether you choose to utilize Azure OpenAI, OpenAI, or other platforms for your models, we ensure a streamlined experience in building your AI app and quickly launching it to Teams.
JavaScript and .NET Languages supported with Python in private preview
The Teams AI Library is now generally available for developers in JavaScript and .NET languages allowing developers to harness the power of AI and create intelligent, user-friendly applications for Microsoft Teams using the programming languages they are most comfortable with. Furthermore, we’re excited to offer a Private Preview for Python developers, opening the door to a broader developer community, eager to build AI-powered Teams apps. With this multi-language support, we are committed to a developer first mindset where you build AI products with the tools and languages you want in order to make the best experiences possible for your customers on Teams.
ActionPlanner
The new ActionPlanner is the main component calling your Large Language Model (LLM) and includes several features to enhance and customize your model. Model plugins simplify configuring your selected LLM to the planner and ships with an OpenAIModel that supports both OpenAI and Azure OpenAI LLMs. Additional plugins for other models like Llama-2 can easily be added, giving you the flexibility to choose what model is best for your use case.
Reliability is dramatically increasing with an internal feedback loop that automatically fixes subpar responses from the LLM. Additionally, the management of token budgets is improved, reducing the chance of exceeding a model’s context window lowering your costs. The ActionPlanner supports augmentation expanding the capabilities and control over your model. Overall, these additions greatly enhance the functionality and robustness of the planning process.
Augmentation
Say goodbye to prompt engineering headaches and say hello to efficiently enhancing and directing your AI model’s responses with Augmentation. When creating a prompt select one of the following Augmentation Modes to tailor your model to your needs, increasing its accuracy and desired outcomes:
- Retrieval Augmented Generation (RAG) – Automatically incorporate real-time, dynamic, and specified external datasources into your model’s responses enabling up to date and contextually accurate results without fine-tuning or re-training your model. Answer questions about today’s sales numbers or customize to a specific user’s data; with RAG your model is no longer stuck in the past. Discover how easy it is to build by trying the RAG bot sample today!
- Monologue – Create AutoGPT-style agents capable of performing multi-step actions independently and reliability with full schema validation and automatic repair included.
- Sequences – Enable your AI assistant to return a sequence of actions for execution with schema validation increasing reliability.
- Functions – Produce structured responses from your model by employing user-defined Functions. These functions are customizable using JSON schemas to define the parameters and their format. The ActionPlanner assesses model responses against the schema, making repairs as needed increasing response reliability and consistency.
Build trust in your AI with enhanced reasoning
With enhanced reasoning, the Teams AI Library tackles the ongoing issue of bot hallucinations head-on by offering an integrated fact-checking system.
When a user interacts with your AI assistant, this system prompts the bot to engage in a process of self-reflection critically evaluating its potential responses prior to sending them. The introspection allows the bot to identify inaccuracies and correct its answers improving accuracy, quality, and contextual relevance. Advanced reasoning ensures that your AI assistant becomes a dependable source of information and judgement, building trust in your product and drawing users back every day.
How do I get started?
Ready to harness the power of the Teams AI library and create your own app’s copilot for Teams?
Get started here:
API message extensions: The fastest way to build message extensions and Copilot plugins
We are excited to announce API message extensions in Microsoft Teams, a new and simple way to build message extensions and, coming soon, Copilot plugins!
Message extensions go beyond the standard out-of-the-box messaging functionalities, allowing users to effortlessly search for and share information using rich cards, capture data, and preview app content directly within Microsoft Teams. This not only enhances the user experience but also streamlines communication and workflows of your apps within Teams.
Leveraging API message extensions to give updates on Azure DevOps tasks
Previously to build a message extension, you were required to use the Bot Framework, which could leverage the full capabilities of a bot but required higher upfront development time. With API message extensions, you are no longer required to register a bot or to use the Bot Framework for message extensions that only communicate with a web service expediting the development process and reducing on-going maintenance. Using Teams Toolkit further simplifies the process, enabling anyone to build an API message extension in as little as 5 minutes!
Beyond the ease of development, API message extensions offer you full control of your data as traffic is privatized since this service does not depend on the Azure bot infrastructure. As all Teams message extensions will be supported as Copilot plugins soon, you can easily extend your app across multiple products within the M365 ecosystem and its hundreds of millions of users.
API message extensions allow Copilot to:
- Retrieve real-time information, for example, latest news coverage on a product launch.
- Retrieve knowledge-based information, for example, my team’s design files in Figma.
- Perform actions on behalf of the user, for example, create a Contoso ticket.
How do I get started?
Bring your existing OpenAPI Definition document and get started with Teams Toolkit today!
Learn More
To learn more about message extensions read the following resource:
Create the new era of AI today
The new era of AI will bring enhanced productivity and collaboration between people and intelligent apps. Bringing your app into Copilot’s AI ecosystem and Teams with API message extensions and infusing AI into your app with the Teams AI Library will deliver the most advanced and engaging user experiences as we transform how workers collaborate with each other and with software.
Our goal at Teams Platform is to make building advanced AI products easy for you. These tools are ready for you to build the AI future as you see it!
Want to learn more about ways to infuse AI into your app on Teams? Join us at Microsoft Ignite (Nov 14-17, 2023). Related Ignite Sessions:
Light
Dark
0 comments