Introducing API Manifest Plugins for Semantic Kernel

Sophia Lagerkrans-Pandey

Mustafa Zengin

{
    "AzureOpenAI": {
        "ChatModelId": "",
        "ChatDeploymentName": "",
        "Endpoint": "",
        "ApiKey": ""
    },
    "MsGraph": {
        "ClientId": "",
        "TenantId": "9188040d-6c67-4c5b-b112-36a304b66dad", // MSA/Consumer/Personal tenant,  https://learn.microsoft.com/azure/active-directory/develop/accounts-overview
        "RedirectUri": "http://localhost"
    }
}

dotnet add package Microsoft.Extensions.Configuration.Json
dotnet add package Microsoft.SemanticKernel.Planners.OpenAI --prerelease 
dotnet add package Microsoft.SemanticKernel.Plugins.MSGraph --prerelease
dotnet add package Microsoft.SemanticKernel.Plugins.OpenAPI.Extensions --prerelease

API Manifest file

Currently, you can author this file manually. In the future, Kiota, our open-source API-client generation tool, will provide the functionality to generate this file from OpenAPI descriptions.

{
    "applicationName": "Message Processor Plugin",
    "description": "This plugin accesses Microsoft Graph API to read emails and GitHub API for Markdown to HTML conversion.",
    "publisher": {
        "name": "Alice",
        "contactEmail": "alice@contoso.com"
    },
    "apiDependencies": {
        "microsoft.graph": {
            "apiDescriptionUrl": "https://raw.githubusercontent.com/microsoftgraph/msgraph-metadata/master/openapi/v1.0/graphexplorer.yaml",
            "requests": [
                {
                    "method": "Get",
                    "uriTemplate": "/me/messages"
                }
            ]
        },
        "github": {
            "apiDescriptionUrl": "https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.yaml",
            "requests": [
                {
                    "method": "POST",
                    "uriTemplate": "/markdown"
                }
            ]
        }
    }
}

• apiDependencies section contains the API dependencies.
• microsoft.graph and github are the names of the dependencies.
• apiDescriptionUrl is the URL of the OpenAPI description.
• requests section contains the requests that are being made by the application.

Sample Program

using Microsoft.Extensions.Configuration;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Planning;
using Microsoft.SemanticKernel.Plugins.MsGraph.Connectors.CredentialManagers;
using Microsoft.SemanticKernel.Plugins.OpenApi;
using Microsoft.SemanticKernel.Plugins.OpenApi.Extensions;

// read config
var configuration = new ConfigurationBuilder()
    .AddJsonFile("appsettings.Development.json", optional: true, reloadOnChange: true)
    .Build();

// initialize kernel
var kernel = Kernel.CreateBuilder()
    .AddAzureOpenAIChatCompletion(
        deploymentName: configuration["AzureOpenAI:ChatDeploymentName"],
        endpoint: configuration["AzureOpenAI:Endpoint"],
        serviceId: "AzureOpenAIChat",
        apiKey: configuration["AzureOpenAI:ApiKey"],
        modelId: configuration["AzureOpenAI:ChatModelId"]).Build();

// prefetch graph token
LocalUserMSALCredentialManager credentialManager = await LocalUserMSALCredentialManager.CreateAsync().ConfigureAwait(false);

var token = await credentialManager.GetTokenAsync(
    configuration["MSGraph:ClientId"],
    configuration["MSGraph:TenantId"],
    // API Manifest file captures required scopes. In the future, we have plans to improve integrations to automate this process
    // so that you don't have to specify the scopes in your auth callback.
    ["Mail.Read"],
    new Uri(configuration["MSGraph:RedirectUri"])).ConfigureAwait(false);

// create graph auth callback to inject auth header
AuthenticateRequestAsyncCallback? graphAuthCallback = (HttpRequestMessage requestMessage, CancellationToken _) => {
    requestMessage.Headers.Add("Authorization", $"Bearer {token}");
    return Task.CompletedTask;
};

// specify auth callbacks for each API dependency
// we don't need one for GitHub as markdown API doesn't require any auth
var apiManifestPluginParameters = new ApiManifestPluginParameters
{
    FunctionExecutionParameters = new ()
    {
        { "microsoft.graph", new OpenApiFunctionExecutionParameters(authCallback: graphAuthCallback) },
        // no need for github authentication as markdown API does not need it
    }
};

// import api manifest plugin
KernelPlugin plugin =
await kernel.ImportPluginFromApiManifestAsync(
    "MessageProcessorPlugin",                   // plugin name
    "MessageProcessorPlugin/apimanifest.json",  // path to api manifest file
    apiManifestPluginParameters)
    .ConfigureAwait(false);

// set goal
var goal = @"
Get my last email, extract the current and previous dates of eclipses, and build the timeanddate eclipse map URL. Then create a Markdown table and get the HTML representation of the Markdown table and output these two tables.

Eclipse map URL is built using the following format: https://www.timeanddate.com/eclipse/map/2024-april-8
";

// create planner
var planner = new FunctionCallingStepwisePlanner(
    new FunctionCallingStepwisePlannerOptions
    {
        MaxIterations = 10,
        MaxTokens = 32000
    }
);

// execute plan
var result = await planner.ExecuteAsync(kernel, goal);

// output result
Console.WriteLine(result.FinalAnswer);

Output in Markdown

| Eclipse Date | Eclipse URL |
|--------------|-------------|
| April 8, 2024 | [View Map](https://www.timeanddate.com/eclipse/map/2024-april-8) |
| December 4, 2021 | [View Map](https://www.timeanddate.com/eclipse/map/2021-december-4) |

Output in HTML (consistent with Markdown)

Rendered Output

• Extracting different types of information from email messages will be only a matter of changing the goal in natural language.
• Generated HTML output can be wired to an email sender with the help of another plugin or with an extension of this plugin with additional endpoints from Graph.
• Generated Markdown output can be backed up to GitHub as a gist.
• and on and on…

Follow-up Challenge

We are looking forward to what you are going to build with API Manifest plugins. Please let us know your feedback!”

From the Semantic Kernel team, we want to thank Mustafa for his time. We’re always interested in hearing from you. If you have feedback, questions or want to discuss further, feel free to reach out to us and the community on the Semantic Kernel GitHub Discussion Channel! We would also love your support, if you’ve enjoyed using Semantic Kernel, give us a star on GitHub.

Sophia Lagerkrans-Pandey

Follow

Mustafa Zengin

Follow