Hello to all Python enthusiasts working with Semantic Kernel! We are very happy to share some of the latest work we have been doing, namely we introduced not one but two new options for templates: Handlebars and Jinja2. And we also made some updates to the ‘semantic-kernel’ (default) template as well! This is all now available as of package version 0.9.3b1.

Templating

Templating is a very core feature of semantic kernel, because it allows you, as the app developer, to define what gets sent to the models. It therefore is one of the most important ways you program against the large language models. The templates serve two purposes. Firstly, it allows you to inject runtime variables into your prompt. Secondly, it allows you to orchestrate “static” functions to be called (as opposed to using a planner for dynamic function calls), for instance this can be a function that does intent recognition on the user input, that changes the casing of texts (if for some reason you want to make the user TALK LIKE THIS), or extend the context in other ways, through search, queries, lookups, etc. It also allows you to decide which things are always sent to a model and what changes overtime. For instance, when creating a application that supports chat interactions, you can decide that the system message is always fixed (or at least tied to the application releases), it is then followed by the chat conversation so far, then a certain instruction to the model is repeated and then the latest user message is added, this could result in a template like this (using ‘semantic-kernel’ templates):

{{$system_message}}{{$chat_history}}{{$repeat_instruction}}{{$user_input}}

and when calling the kernel to do the completion, the command is this:

kernel.invoke(
    chat_function, 
    system_message=system_message, 
    repeat_instruction=repeat_instruction, 
    chat_history=chat_history, 
    user_input=user_input
)

and in this the last two parameters are updated as the application is used, while the first two are default strings. One note on this example is that there are a number of ways to achieve this same thing, like this template:

you are an intelligent assistant, you are designed to ... {{$chat_history}} remember to respond with ... {{$user_input}}

and you can even do this: {{$chat_history}} where you use code to update the chat_history in the right way!

The reason all of these templates can be used to create the same thing is that under the covers in the python SDK we parse these templates into a intermediate structure which is a mix of plain text and XML, this looks something like this:

system prompt<chat_history><message role="user">Hi, my name is Eduard</message></chat_history>I want to know more about Semantic Kernel!

If the service used is a text completion service this gets sent to the model directly, while for a chat model, this will get parsed back into a single ChatHistory object, and that object will then be parsed to a dictionary that the model expects. The format is different for instance for Google models versus OpenAI models. Knowing about this structure means you can also leverage the same things, by manually inputting the same xml tags into your prompt, additional fields of messages, especially for OpenAI are represented in this format as well, as part of the fields within the message tag, so a tool call response message would look like this: <message role=tool tool_call_id=call_123876>tool response</message>.

On to what’s new!

Handlebars Templating

The first new template format is Handlebars. Handlebars was created to be a lightweight templating language “on steroids” (their words), it is already available in the .NET SDK. The reason we wanted to include new templating approaches is because there are some more advanced, but well known structures that these language support out of the box, like loops, conditionals and other functions, for instance with a chat_history object (which is iterable) we can now define a template like this:

`{{#each chat_history}}{{#message role=role}}{{~content~}}{{/message}} {{/each}}`

this will render each message to a xml object with the role and content in it. The way this works is that there are a number of helpers built-in to the package, some are added by us for Semantic Kernel and all the functions that are registered in the kernel are also exposed as functions. Some of the differences between the semantic-kernel templates and handlebars, are that in handlebars there is no distinction between variables and values (in semantic-kernel a variable is denoted with a $-prefix, while a value is marked with single quotes), in handlebars everything between a handlebar (the double brackets) is looked up against the helpers and the parameters passed in through KernelArguments, and if it calls a helper then that function is executed, while if it is a variable name in arguments that is used, and if it is neither then the text itself is returned, so the template {{input}} results in either the text ‘input’ or the value of the argument called ‘input’ that is passed in KernelArguments. If you add arguments into a handlebar, like this {{add 1 2}} then it will look for a function called ‘add’ that is can send these parameters to. To use this template language, you need to set the parameter ‘template_format’ to ‘handlebars’ in your PromptTemplateConfig or by using the kernel.create_function_from_prompt method Down below a full list of all helpers and their existence in the different template languages is shown.

Jinja2 Templating

Next up is Jinja2, this is a fast and powerful templating language for Python, designed to generate dynamic content efficiently. Its templating system allows developers to use variables, loops, and conditional statements, enabling the creation of complex and dynamic prompts. Its extensibility facilitates the integration of custom filters and functions, allowing for precise data preprocessing and formatting to meet specific needs. We’ve enriched the environment with numerous helper functions, including those for interacting with chat history objects (such as message_to_prompt and message), alongside utilities for arrays, ranges, concatenation, and logical operations. Additionally, Jinja2 underscores the importance of security with automatic variable escaping, effectively guarding against injection attacks and ensuring the safety of dynamically generated text. It’s important to note that, in leveraging Python’s capabilities, Jinja2 adapts to Python’s syntax conventions. Therefore, the usual fully qualified names for Semantic Kernel plugins and functions, typically expressed with hyphens (plugin-function), should be adapted to Python’s naming requirements by using underscores (e.g., plugin_function). This adjustment ensures seamless integration and access to registered plugins and their functionalities, broadening the scope of what can be achieved with your SDK.

The Jinja2 logoLet’s look at an example of how one can craft a Jinja2 prompt that operates on the application’s chat history. For brevity, some of the code is ommitted. You can find the full example as part of our kernel samples titled “azure_chat_gpt_api_jinja2.py.”

chat_function = kernel.create_function_from_prompt(
    prompt="""{{system_message}}{% for item in chat_history %}{{ message(item) }}{% endfor %}""",
    function_name="chat",
    plugin_name="chat",
    template_format="jinja2",
    prompt_execution_settings=req_settings,
)

chat_history = ChatHistory(system_message="You are a helpful chatbot.")
chat_history.add_user_message("User message")
chat_history.add_assistant_message("Assistant message")

After rendering the prompt, we have the following text:

Conclusion

In conclusion, we feel that these additional capabilities will give developers even more tools to do great things with Semantic Kernel in ways that make sense to many of you from other efforts. If you have comments, or would like to see additional helpers feel free to create a issue on the repo, or post something in the Discord for Semantic Kernel!

Available helpers

Note that this list is no exhaustive. Jinja2 comes with packed with built-in filters, tests, and functions. For more information about what is available in the package, please visit Jinja2’s documentation: built-in filters, built-in tests, built-in functions.

Summary

Please reach out if you have any questions or feedback through our Semantic Kernel GitHub Discussion Channel. We look forward to hearing from you! We would also love your support, if you’ve enjoyed using Semantic Kernel, give us a star on GitHub.