Using Handlebars Planner in Semantic Kernel

Sophia Lagerkrans-Pandey

Teresa Hoang

We are excited to dive into the Handlebars Planner, a powerful tool for creating customized plans.

This comprehensive guide will provide you with a thorough understanding on how to create your own plans using the Handlebars Planner and explain some of the planner options you can customize, such as providing additional information when creating the prompt. This guide provides sample code and instructions on how to instantiate the kernel, import plugins, specify planner options, create and invoke the plan, and handle error scenarios.

The Handlebars Planner uses the Handlebars syntax to generate a plan, allowing the model to leverage native features like loops and conditions without additional prompting

Running the Planner and Handling Error Scenarios

1. Instantiate the kernel and import plugins

Instantiate the kernel and import any plugins you need using the Kernel.CreateBuilder()method.

var kernel = Kernel.CreateBuilder()
  .AddAzureOpenAIChatCompletion(
    deploymentName: "gpt-4",
    endpoint: <Azure OpenAI Endpoint>,
    serviceId: "AzureOpenAIChat",
    apiKey: <Azure OpenAI API Key>,
    modelId: "gpt-4"
  ).Build();

await kernel.ImportPluginFromOpenApiAsync(
  PluginName,
  new Uri("https://<url_to_plugin_manfest>/openapi.yaml"));

2. Specify any planner options you want

Next, specify the options for the HB Planner using the HandlebarsPlannerOptions class. For example, to allow loops in your plan, you can use the following code.

Note: Use gpt-4 or newer models if you want to test with loops. Older models like gpt-35-turbo are less recommended. They do handle loops but are more prone to syntax errors.

var plannerOptions = new HandlebarsPlannerOptions()
    {
        // When using OpenAI models, we recommend using low values for temperature and top_p to minimize planner hallucinations.
        ExecutionSettings = new OpenAIPromptExecutionSettings()
        {
            Temperature = 0.0,
            TopP = 0.1,
        },
        // Use gpt-4 or newer models if you want to test with loops.
        // Older models like gpt-35-turbo are less recommended. They do handle loops but are more prone to syntax errors.
        AllowLoops = ChatDeploymentName.Contains("gpt-4", StringComparison.OrdinalIgnoreCase),
    };

3. Instantiate the planner and create the plan

With the kernel and planner options configured, you can now instantiate the planner and create a plan.

To create a plan for the goal “Find a restaurant near me”, you can use the following code.

Note: Always wrap your`CreatePlanAsync`call in a try catch block if you want to inspect data about the prompt and LLM response. This follows standard exception patterns found across SK.

If the planner fails to create a plan for any reason, it will throw a`PlanCreationException.`For certain known failure cases, the exception will contain an inner`KernelException`with a specified`HandlebarsPlannerErrorCode`. All`PlanCreationExceptions`will contain the prompt used to create the plan, as well as the LLM results (the plan proposed by the model).

try {
  var planner = new HandlebarsPlanner(plannerOptions);
  var plan = await planner.CreatePlanAsync(kernel, goal);
  var planResult = await plan.InvokeAsync(kernel);
}
catch (PlanCreationException e)
{
  var errorDetails = e.InnerException?.Message;
  var promptDetails = e.CreatePlanPrompt;
  var modelResults = exception?.ModelResults?.Content; // Proposed plan
}

4. Invoke the plan

Finally, you can invoke the plan using the`InvokeAsync` method.

var planResult = await plan.InvokeAsync(kernel);

Planner Options

Now that you know how to create and invoke a Handlebars Plan, we’ll talk you through the customizable options available for the Handlebars Planner.

These options allow you to tailor the planner to your specific needs, including the ability to:

• Override the default CreatePlan prompt with your own prompt
• Provide additional context or rules for the planner
• Use predefined arguments when invoking the plan

By taking advantage of these options, you can create plans that are more specific to your use case and improve the performance and accuracy of your planner.

1. CreatePlanPromptHandler

This option allows you to override the default CreatePlan prompt with your own prompt. This is useful if you want to provide a prompt that is more tailored to your use case or domain. You can even use Handlebars syntax to generate the prompt, allowing you to leverage native features like loops and conditions without additional prompting.

Building a custom prompt

When creating your custom prompt, you have access to all of the partials located in the CreatePlanPromptPartials folder. These partials can be referenced in your custom prompt using Handlebars syntax. For any other custom content, be sure to use the ChatHistory syntax so the completion request is formatted correctly before it’s sent to the model.

For example, let’s say you want to create a custom prompt that instructs users to create a plan using only the built-in loop helpers. You can create a handlebars file that looks like this:

{{!-- Example of a custom CreatePlan prompt for the Handlebars Planner.  --}}
{{#message role="system"}}
Explain how to achieve the users goal using the available helpers with a Handlebars .Net template.
{{~/message}}

{{> VariableHelpers }}
{{> LoopHelpers }}
{{> UserGoal }}
{{> TipsAndInstructions }}

In this example, we have pulled in the built-in ` VariableHelpers`,` LoopHelpers`,`UserGoal`*, and`TipsAndInstructions`partials from the CreatePlanPromptPartials folder, and included a custom message introductory block.

Note: that you can inject the user goal manually by referencing the variable`{{goal}}`.

Using our built-in partials allows you to leverage SK’s templating and prompt engineering. We recommend always using the`{{> UserGoal}}`and` {{> AdditionalContext }}`partials in your custom prompt to ensure all user and domain context is appropriately populated as well as the`{{> CustomHelpers}}`and`{{> VariableHelpers}}`partials to populate all kernel function definitions and allow for state manipulation.

Using the CreatePlanPromptHandler option

Once you have created your custom prompt, you can provide it as a delegate that returns a string to the`CreatePlanPromptHandler`option. Make sure to load any additional helpers or partials your custom prompt requires.

var plannerOptions = new HandlebarsPlannerOptions()

{
    // Context to be used in the prompt template.
    GetAdditionalPromptContext = () => 
    Task.FromResult("If the goal cannot be fully achieved with the provided helpers, call the `\\{Plugin.CustomFunctionName}` helper."),
};

Please note that setting the CreatePlanPromptHandler directly as a string will not work. You must specify a delegate that returns a string or file stream. We configured the option this way to allow use cases such as reading in a custom prompt from a file as shown above.

// Invalid
var plannerOptions = new HandlebarsPlannerOptions()

{

    CreatePlanPromptHandler = "Create a Handlebars template to ...",

};

// Valid

var plannerOptions = new HandlebarsPlannerOptions()

{

    CreatePlanPromptHandler = () => "Create a Handlebars template to ...",

};

You can run an E2E sample using a custom prompt at Example65_HandlebarsPlanner.RunOverrideCreatePlanPromptSampleAsync. This sample loads a custom prompt from the 65-prompt-override.handlebars file, which mixes the partial referencing described above as well as custom content.

With this option in place, the Handlebars Planner will use your custom prompt instead of the default prompt when creating plans.

2. GetAdditionalPromptContext

If you want to provide additional context or rules to the planner that can help improve its performance and accuracy, you can use the`GetAdditionalPromptContext`option.

This option allows you to pass in any domain knowledge or specific content that might be helpful for the model to fulfill the goal. This context should just hold static information that’s the same for every request. It should not include or define any variables (use KernelArguments when invoking the planner instead).

Like the`CreatePlanPromptHandler`option, this option expects a delegate that returns a string.

var plannerOptions = new HandlebarsPlannerOptions()
{
    // Context to be used in the prompt template.
    GetAdditionalPromptContext = () => Task.FromResult("If the goal cannot be fully achieved with the provided helpers, call the `\\{Plugin.CustomFunctionName}` helper."),
};

Again, assigning a string directly won’t work.

// Invalid
var plannerOptions = new HandlebarsPlannerOptions()
{
    // Context to be used in the prompt template.
    GetAdditionalPromptContext = "If the goal cannot be fully achieved with the provided helpers, call the `\\{Plugin.CustomFunctionName}` helper.",
};

Please note that this option expects an asynchronous return type to accommodate handlers that need to fetch data. If your handler doesn’t require asynchronous functionality, you can simply wrap the result in a Task.FromResult, as demonstrated in the example above.

3. Using Predefined Arguments

If you want to run a Handlebars plan with a specific set of argument values, you can define an initial set of`PlanCreationExceptions`to pass into the Planner.

var initialArguments = new KernelArguments()
{
    { "greetings", new List<string>(){ "hey", "bye" } },
    { "someNumber", 1 },
    { "person", new Dictionary<string, string>()
      {
          {"name", "John Doe" },
          { "language", "Italian" },
      }
    }
 };

var planner = new HandlebarsPlanner(plannerOptions);

It’s important to note that if you use a set of predefined arguments when creating the plan, you must also pass in these arguments when invoking the plan.

var plan = await planner.CreatePlanAsync(kernel, goal, initialArguments);

var planResult = await plan.InvokeAsync(kernel, initialArguments);

You can run the plan with a different set of argument values if all the required parameters are present and their values are typed correctly.

var newArgs = new KernelArguments()
{
  { "greetings", new List<string>(){ "hola", "adiós", "hasta pronto", "hasta luego" } },
  { "someNumber", 19 },
  { "person", new Dictionary<string, string>()
    {
      {"name", "Jane Doe" },
      { "language", "Spanish" },
    }
  }
};

var planResult = await plan.InvokeAsync(kernel, newArgs);

Templatizing a HandlebarsPlan

If you want to save a Handlebars plan for later use or share it with others, you can templatize it by serializing it into a string.

This way, you can load the plan into a new HandlebarsPlan object and run it at a later time. In this section, we’ll show you two options for templatizing a Handlebars plan and provide some examples of how to use them.

Option 1: Serializing the entire Plan object

var plan = await planner.CreatePlanAsync(kernel, goal, initialContext);

// Serialize plan and save offline

var savedPlan = JsonSerializer.Serialize(plan);

// Load plan and deserialize into a HandlebarsPlan object

var deserializedPlan = JsonSerializer.Deserialize<HandlebarsPlan>(savedPlan);

// Remember to pass in arguments if you used any pre-defined variables when creating the plan.

var result = await deserializedPlan.InvokeAsync(kernel, initialContext);

Option 2: Saving only the plan template

• Option 1 is preferred as you retain all the creation states (i.e., CreatePlan prompt) of the plan. But if you don’t need that, this option will save on storage.

var plan = await planner.CreatePlanAsync(kernel, goal, initialContext);

// You can also just save the plan template and load it into a plan object at a later time

// Save this string somewhere offline

var planTemplate = plan.ToString();

// Load plan and deserialize into a HandlebarsPlan object

var savedPlan = new HandlebarsPlan(planTemplate );

// Be sure to include any necessary arguments if you used any pre-defined variables when creating the plan.

var result = await savedPlan.InvokeAsync(kernel, initialContext);

So there you have it, the Handlebars Planner! It’s a cool tool that lets you create customized plans using the Handlebars syntax and native features like loops and conditions to make plans more specific to your use case.

Have fun exploring the Handlebars Planner and all its possibilities!

Dive Deeper

We recommend checking out this article for information on the Planners here.

We’re always interested in hearing from you. If you have feedback, questions or want to discuss further, feel free to reach out to us and the community on the Semantic Kernel GitHub Discussion Channel! We would also love your support, if you’ve enjoyed using Semantic Kernel, give us a star on GitHub.

Sophia Lagerkrans-Pandey

Follow

Teresa Hoang

Follow