October 28th, 2024

Diving into Function Calling and its JSON Schema in Semantic Kernel .NET

Diving into Function Calling and its JSON Schema in Semantic Kernel .NET

Function-callingĀ is one of the most exciting features of certain Large Language Models (LLMs), enabling developers to execute code directly in response to user queries. In Semantic Kernel, we streamline the process by allowing you to use built-in plugins or integrate your own code with ease. Today, weā€™ll explore how Semantic Kernel creates a function-calling JSON schemaā€”a critical component that helps the model determine which function to invoke in various contexts.

For those unfamiliar, function-calling refers to running local code, often on a user’s machine or within their deployment, to satisfy a user’s query to an LLM. If youā€™ve ever asked an LLM to perform math calculations without the code interpreter, you might have noticed it sometimes produces inaccurate results. Since LLMs generate responses based on probabilistic models, the outcome may not always be correct for operations like large number addition (e.g., 102982 + 2828381). To handle such deterministic tasks more accurately, we use function calling. For further reading, check out OpenAI’sĀ function calling documentation.

Semantic Kernel provides abstractions over various LLMs. When a user configures their execution settings withĀ functionChoiceBehavior = FunctionChoiceBehavior.Auto();, the SDK includes a special JSON payload with aĀ toolsĀ attribute in the request settings. Weā€™ll explore different plugin/function configurations and the JSON payloads they generate.

A Simple Example: Adding Two Numbers

Let’s start with a simple example of adding two numbers. Hereā€™s a Semantic Kernel (SK) plugin/function:

/// <summary>
/// A sample Math Plugin.
/// </summary>
[KernelFunction, Description("Adds two numbers together and provides the result")]
[return: Description("The result of adding the two numbers")]
public int AddNumbers(
    [Description("The first number to add")] int numberOne,
    [Description("The second number to add")] int numberTwo)
{
    // A sample Semantic Kernel Function to add two numbers.
    return numberOne + numberTwo;
}

Once the C# code is compiled and executed, the Semantic Kernel SDK scans for functions marked with theĀ [KernelFunction]Ā attribute. It parses these functions for their names, input parameters, and output types, which are stored inĀ KernelParameterMetadata. This metadata is crucial for building the JSON schema the model uses to call the correct function.

TheĀ DescriptionĀ attributes help provide context to the model about the functionā€™s purpose and the expected inputs and outputs.

JSON Schema for the AddNumbers Plugin

As mentioned, the information parsed from a kernel function is stored in KernelParameterMetadata. When constructing the JSON schema, we classify the parameter typeā€”integer, string, boolean, or object.

Hereā€™s the JSON schema for the AddNumbers plugin:

{
    "type": "function",
    "function": {
        "name": "Math_AddNumbers",
        "description": "Adds two numbers together and provides the result",
        "parameters": {
            "type": "object",
            "properties": {
                "numberOne": {
                    "type": "integer",
                    "description": "The first number to add"
                },
                "numberTwo": {
                    "type": "integer",
                    "description": "The second number to add"
                }
            },
            "required": [
                "numberOne", "numberTwo"
            ]
        }
    }
}

The schema specifies that bothĀ numberOneĀ andĀ numberTwoĀ are required integer inputs. This schema enables the model to recognize which parameters are necessary for this function call.

A More Complex Example: Working with Complex Types

Letā€™s explore a more advanced example with complex data types. First, weā€™ll define a C# model classĀ ComplexRequest, which contains two properties –Ā StartDateĀ andĀ EndDate. These properties are annotated withĀ DataAnnotationsĀ attributes to indicate that they are required and to provide descriptions and example values. You can refer to theĀ DataAnnotationsĀ for more details on using these annotations in C#.

Defining the Model Class

public class ComplexRequest
{
    [Required]
    [Display(Description = "The start date in ISO 8601 format")]
    public string StartDate { get; set; }

    [Required]
    [Display(Description = "The end date in ISO 8601 format")]
    public string EndDate { get; set; }

    // Example values
    public static readonly string[] StartDateExamples = { "2023-01-01", "2024-05-20" };
    public static readonly string[] EndDateExamples = { "2023-01-01", "2024-05-20" };
}

Next, we define a plugin namedĀ ComplexTypePlugin, containing a functionĀ BookHolidayĀ that accepts aĀ ComplexRequestĀ object.

Creating the Plugin

public class ComplexTypePlugin
{
    [KernelFunction, Description("Answer a request")]
    [return: Description("The result is the boolean value True if successful, False if unsuccessful.")]
    public bool BookHoliday 
            ([Description("A request to answer.")] 
             ComplexRequest request)
    {
        return true;
    }
}

This kernel functionā€™s input parameterĀ requestĀ is of typeĀ ComplexRequest. When the SDK parses this function, it recurses through theĀ ComplexRequestĀ class to identify the required properties, likeĀ StartDateĀ andĀ EndDate, for inclusion in the JSON schema.

JSON Schema for Complex Types

Hereā€™s the generated JSON schema:

{
    "type": "function",
    "function": {
        "name": "complex-book_holiday",
        "description": "Answer a request",
        "parameters": {
            "type": "object",
            "properties": {
                "request": {
                    "type": "object",
                    "properties": {
                        "StartDate": {
                            "type": "string",
                            "description": "The start date in ISO 8601 format"
                        },
                        "EndDate": {
                            "type": "string",
                            "description": "The end date in ISO 8601 format"
                        }
                    },
                    "required": [
                        "StartDate",
                        "EndDate"
                    ],
                    "description": "A request to answer."
                }
            },
            "required": [
                "request"
            ]
        }
    }
}

The schema breaks down theĀ ComplexRequestĀ object into its individual properties, ensuring that the required fieldsĀ StartDateĀ andĀ EndDateĀ are validated before the function call proceeds.

Example Interaction

When this plugin is used in a model, you might see an interaction like this:

User: > Answer a request for me.
Assistant: > Certainly! Please provide me with the start date and end date for your request, and I shall endeavor to assist you with it!

I can then respond with:

User: > The start date is Feb 10, 2023 to Mar 10, 2024.
Assistant: > The request has been successfully answered with a resounding "True"! If there are any further inquiries or additional assistance you seek, do not hesitate to ask!

Youā€™ll notice that I didnā€™t need to format my exact dates in ISO-8601 format. We can tell the model how to format the dates, and it will follow along to the best of its ability. The following dates are what the model returned as part of the function call arguments:

request.start_date = '2023-02-10'
request.end_date = '2024-03-10'

Then, our plugin code that expects the ISO-8601 format can proceed without a hitch.

Conclusion

In summary, we’ve explored function calling within the Semantic Kernel SDK, focusing on JSON schema construction and the importance of parameter communication. To sum up:

  1. We can provide string descriptions via theĀ KernelFunctionĀ attributeā€™sĀ DescriptionĀ parameter and input parameter descriptions to help give the model more context about our function or parameters.
  2. We can define complex objects that use underlying complex type objects, along with annotations or models to give further information and context to the model, which can aid it in choosing the correct function to invoke to complete the userā€™s query.

For complete implementations around handling auto function calling in dotnet/C#, please see our code samplesĀ FunctionCalling in dotnet.

As we continue bringing you new features in Semantic Kernel, we want to highlight that we are tracking anĀ issueĀ related adding support for OpenAIā€™s structured outputs. If youā€™re looking for a challenge on the C# side and want to help us out with this, please feel free to comment on the open issue weā€™re tracking or create aĀ GitHub discussionĀ to let us know youā€™d like to assist with the integration. Weā€™ll be happy to work with you.

Thank you for your time, and we welcome any feedback related to this post or Semantic Kernel in general.

0 comments