In the world of software development, flexibility and adaptability are key. Developers often face challenges when it comes to updating plugin metadata dynamically without disrupting services or requiring redeployment. This is where SemanticPluginForge, an open-source project, steps in to improve the way we manage plugin metadata.

LLM Function Calling Feature

The function calling feature in LLMs allows developers to define a set of functions that the model can invoke during a conversation. These functions are described using metadata, which includes the function name, parameters, and their descriptions. The LLM uses this metadata to determine when and how to call a function, ensuring seamless integration between the model and external systems.

Semantic Kernel Plugins and Function Calling

Semantic Kernel plugins enhance the LLM function-calling experience by providing a structured way to define and manage these functions. Each plugin encapsulates a set of related functionalities, exposing them through well-defined metadata. This metadata includes descriptions, parameter details, and return value information, making it easier for developers and AI systems to understand and utilize the functions effectively.

SemanticPluginForge builds on this foundation by offering tools to dynamically manage and update plugin metadata, ensuring that function calling remains flexible, adaptable, and user-friendly.

The Pain Points

1. Static Metadata Management

Traditionally, plugin metadata—such as descriptions, parameter details, and return value information—is hardcoded or statically defined. This approach creates several challenges:

• Downtime for Updates: Any change to metadata often requires redeployment, leading to service interruptions.
• Limited Customization: Adapting metadata to evolving business needs or user feedback becomes cumbersome.
• Rigid Architecture: Static metadata lacks the flexibility to support dynamic use cases or future expansions.

2. Inconsistent User Experience

When metadata cannot be updated dynamically, it may lead to outdated or irrelevant descriptions, confusing users and reducing the overall effectiveness of the plugins.

3. Complex Maintenance

Managing metadata across multiple plugins and ensuring consistency can become a maintenance challenge, especially in large-scale systems.

4. Challenges with OpenAPI Specifications

When using an OpenAPI specification for plugins, additional challenges arise:

• Lack of Control: Often, developers do not have control over the API to update descriptions and make them friendly for large language models (LLMs).
• Change Management Overhead: Even if control exists, applying changes may require a significant change management process.
• Balancing Human and LLM Needs: Descriptions must be crafted to be equally understandable by humans and LLMs, which can be a complex task.
• Local Copy Maintenance: Keeping a local copy of the OpenAPI spec to make adjustments is another option, but it is not ideal as it introduces redundancy and potential inconsistencies.

5. Wrapper Classes for SDKs

For existing classes, such as those from an SDK like a home automation library, developers often need to wrap the class in a custom class and then decorate it using attributes in the wrapper. This process adds unnecessary complexity and overhead, making it harder to integrate such classes into the plugin ecosystem.

Enter SemanticPluginForge

SemanticPluginForge is designed to address these pain points by introducing a dynamic and extensible approach to plugin metadata management. Here’s how it can help:

1. Dynamic Metadata Updates

With SemanticPluginForge, you can make real-time updates to plugin metadata without redeploying your application. This ensures:

• Zero Downtime: Update descriptions, parameters, and return values seamlessly.
• Enhanced Flexibility: Quickly adapt to changing requirements or user feedback.

2. Extensible Architecture

The library provides an interface, IPluginMetadataProvider, that allows developers to implement custom metadata providers. For example:

• Fetch metadata from a database or remote service.
• Customize metadata based on specific business logic.

3. Suppressing Functions and Parameters

SemanticPluginForge introduces the ability to suppress specific functions or parameters in plugin metadata. This feature is particularly useful for:

• Hiding sensitive or irrelevant details from end-users.
• Maintaining functionality while controlling visibility.

4. Improved User Experience

By dynamically tuning plugin metadata, you can ensure that users always have access to accurate and relevant information, leading to a more intuitive and effective experience.

5. Simplified Maintenance

Centralized and dynamic metadata management reduces the complexity of maintaining consistency across plugins, saving time and effort.

6. Future-Proof Design

SemanticPluginForge is built with scalability in mind, ensuring that it can handle the growing demands of modern applications. Its modular design allows for seamless integration with new technologies and frameworks.

7. Simplified Integration for Existing Classes

With SemanticPluginForge, you can create a plugin out of any class or object without the need for wrapping it in a custom class or decorating it with attributes. Instead, you only need to provide the necessary metadata, enabling seamless integration and allowing these classes to be leveraged effectively in your agents.

How to Get Started

Using SemanticPluginForge is straightforward. Here’s a quick overview:

1. Download the Package: Install the SemanticPluginForge package via NuGet using the following command:

   dotnet add package SemanticPluginForge.Core
   

2. Implement a Custom Metadata Provider: Define your own logic for updating metadata dynamically by implementing the IPluginMetadataProvider interface.

public class SampleMetadataProvider : IPluginMetadataProvider
{
    public FunctionMetadata? GetFunctionMetadata(KernelPlugin plugin, KernelFunctionMetadata metadata) =>
        plugin.Name switch
        {
            "WeatherPlugin" => metadata.Name == "GetTemperatureByCity" ? new FunctionMetadata(metadata.Name)
            {
                Description = metadata.Description,
                Parameters = [
                    new ParameterMetadata("name")
                    {
                        Description = "The name of the city should be retrieved from the user context, if not in context, please ask the user.",
                        IsRequired = true,
                    },
                    new ParameterMetadata("unit")
                    {
                        Description = "This description does not matter as this will always be suppressed and the default will be used.",
                        IsRequired = false,
                        Suppress = true,
                        DefaultValue = "celsius",
                    }
                ],
                ReturnParameter = new ReturnParameterMetadata
                {
                    Description = "The temperature of the city in the specified unit."
                },
            }
            : null,
            "ShortDatePlugin" => metadata.Name == "ToShortDateString" ? new FunctionMetadata(metadata.Name)
            {
                Description = "Returns the date in short format."
            } : null,
            _ => null,
        };

    public PluginMetadata? GetPluginMetadata(KernelPlugin plugin) =>
        plugin.Name switch
        {
            "WeatherPlugin" => new PluginMetadata { Description = "This plugin can be used to retrieve information about the weather." },
            "ShortDatePlugin" => new PluginMetadata { Description = "This plugin returns date and time information." },
            _ => null,
        };
}

3. Register the Metadata Provider: Add your custom provider to the service collection to enable dynamic updates.

services.AddSingleton<IPluginMetadataProvider, CustomMetadataProvider>();

4. Add Plugins with Patched Metadata: Use the provided extension methods to integrate plugins with updated metadata into your application.

kernel.Plugins.AddFromTypeWithMetadata<WeatherServicePlugin>("WeatherService");

5. Add types without KernelFunction attributes defined: Use the provided extension methods to integrate plugins with updated metadata into your application.

kernelBuilder.Plugins.AddFromClrObjectWithMetadata(new DateTime(), "ShortDatePlugin");

For detailed examples and code snippets, check out the project’s README.

Join the Community

SemanticPluginForge is open-source and welcomes contributions from developers worldwide. Whether you want to report an issue, suggest a feature, or submit a pull request, your input is valuable. Join the discussion on our GitHub repository and help shape the future of dynamic metadata management.

Conclusion

SemanticPluginForge helps developers overcome the limitations of static metadata management, offering a dynamic, extensible, and user-friendly solution. By adopting this library, you can enhance the flexibility, maintainability, and overall effectiveness of your plugins.

Ready to improve the way you manage plugin metadata? Explore SemanticPluginForge today and take your development to the next level!