{"id":140,"date":"2023-03-30T08:08:02","date_gmt":"2023-03-30T15:08:02","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/semantic-kernel\/?p=140"},"modified":"2023-03-31T04:53:30","modified_gmt":"2023-03-31T11:53:30","slug":"semantic-kernel-planner-improvements-with-embeddings-and-semantic-memory","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/agent-framework\/semantic-kernel-planner-improvements-with-embeddings-and-semantic-memory\/","title":{"rendered":"Semantic Kernel Planner: Improvements with Embeddings and Semantic Memory"},"content":{"rendered":"<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2023\/03\/skpatternlarge-1536x128.png\" \/><\/p>\n<p>Semantic Kernel (SK) is a lightweight SDK that lets you mix conventional programming languages, like C# and Python, with the latest in Large Language Model (LLM) AI \u201cprompts\u201d with prompt templating, chaining, and planning capabilities.\u00a0 Its <a href=\"https:\/\/aka.ms\/sk\/concepts\/planner\">Planner<\/a> Skill allows users to create and execute plans based on semantic queries. Recently, the addition of embeddings exposed via <code>SemanticTextMemory<\/code> has made the Planner Skill even more versatile.<\/p>\n<p>In this blog post, we will discuss how integrating embeddings into the Planner Skill to determine related skills enhances its usability and functionality.<\/p>\n<h2><strong>What It Does<\/strong><\/h2>\n<p>When you create a Plan with <a href=\"https:\/\/github.com\/microsoft\/semantic-kernel\/blob\/main\/dotnet\/src\/SemanticKernel\/CoreSkills\/PlannerSkill.cs\">Planner Skill<\/a> today, it uses all registered functions to compose a natural language manual. However, when the number of functions and skills grows, it can hinder the quality of the generated results. By enabling semantic filtering, Plan creation will use Semantic Memory to store embeddings for registered functions. It will then search the Semantic Memory Store based on the Plan goal to filter what functions are used when generating the natural language manual.<\/p>\n<p><em>Additionally, users can limit the number of functions returned from semantic filtering, specify skills and function to exclude and specify functions to include regardless of their semantic relevance.<\/em><\/p>\n<h3><strong>Enable Semantic Filtering with Planner Skill<\/strong><\/h3>\n<p>First, to support Semantic Filtering with <a href=\"https:\/\/github.com\/microsoft\/semantic-kernel\/blob\/main\/dotnet\/src\/SemanticKernel\/CoreSkills\/PlannerSkill.cs\">Planner Skill<\/a>, you need to register an embeddings backend and include an <code>IMemoryStore<\/code> implementation with the kernel used for Planning. For example:<\/p>\n<pre class=\"prettyprint language-cs language-csharp\"><code class=\"language-cs language-csharp\">var kernel = new KernelBuilder()\r\n\u00a0 \u00a0 .Configure(\r\n\u00a0 \u00a0 \u00a0 \u00a0 config =&gt;\r\n\u00a0 \u00a0 \u00a0 \u00a0 {\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 config.AddAzureOpenAITextCompletion(...);\r\n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 <strong>config.AddAzureOpenAIEmbeddingGeneration(...);<\/strong>\r\n\u00a0 \u00a0 \u00a0 \u00a0 })\r\n\u00a0 \u00a0 <strong>.WithMemoryStorage(new VolatileMemoryStore())<\/strong>\r\n\u00a0 \u00a0 .Build();<\/code><\/pre>\n<p>Then, to enable Semantic Filtering when using the Planner Skill, you simply need to supply <code>ContextVariable<\/code> for setting a relevance threshold when creating a plan. When you do this, upon creation of a plan using the Planner Skill, the <code>IMemoryStore<\/code> implementation associated with the kernel instance will be used to search for relevant functions.<\/p>\n<pre class=\"prettyprint language-cs language-csharp\"><code class=\"language-cs language-csharp\">var planner = kernel.ImportSkill(new PlannerSkill(kernel), \"planning\");\r\n\r\n\/\/ import any other skills or functions ....\r\n\r\nvar context = new ContextVariables(\"Create a book with 3 chapters about a group of kids in a club called 'The Thinking Caps.'\");\r\n\r\n\/\/ Set the RelevancyThreshold to enable semantic filtering\r\ncontext.Set(PlannerSkill.Parameters.RelevancyThreshold, \"0.78\");\r\n\r\nvar results = await kernel.RunAsync(context, planner[\"CreatePlan\"]);<\/code><\/pre>\n<p>Lastly, the other configuration values can be set like this:<\/p>\n<pre class=\"prettyprint language-cs language-csharp\"><code class=\"language-cs language-csharp\">\/\/ To limit the number of relevant functions to include\r\ncontext.Set(PlannerSkill.Parameters.MaxRelevantFunctions, \"5\");\r\n\r\n\/\/ To exclude certain functions or entire Skills from being included in plan creation requests\r\ncontext.Set(PlannerSkill.Parameters.ExcludedFunctions, \"NovelChapter\");\r\ncontext.Set(PlannerSkill.Parameters.ExcludedSkills, \"email\");\r\n\r\n\/\/ Or to ensure specific functions are included in plan creation requests regardless of semantic filtering\r\ncontext.Set(PlannerSkill.Parameters.IncludedFunctions, \"DadJoke\");<\/code><\/pre>\n<p>So, there you have it! With the integration of embeddings and Semantic Memory, the Planner Skill in the Semantic Kernel just got even cooler. It&#8217;s like giving your conversational agent a superpower &#8211; the ability to understand context and generate more accurate plans. Follow along for a closer look at how the Planner does this using Semantic Memory.<\/p>\n<h3><strong>How It Works<\/strong><\/h3>\n<p>First, all registered functions need to be saved to memory. When <code>CreatePlan<\/code> is called, we use the <code>SKContext<\/code> as context to retrieve pointer to the <code>Memory<\/code> instance and call <code>SaveInformationAsync<\/code> with known registered functions in the <code>SkillCollection<\/code> on the<code>SKContext<\/code>.<\/p>\n<pre class=\"prettyprint language-cs language-csharp\"><code class=\"language-cs language-csharp\">\/\/ Loop through available functions and save them to memory.\r\nforeach (var function in availableFunctions)\r\n{\r\n    var functionName = function.ToFullyQualifiedName();\r\n    var key = string.IsNullOrEmpty(function.Description) ? functionName : function.Description;\r\n\r\n    await context.Memory.SaveInformationAsync(PlannerMemoryCollectionName, key, functionName, function.ToManualString(),\r\n        context.CancellationToken);\r\n}<\/code><\/pre>\n<p>Then, we can use the goal text as a semantic query against the <code>Memory<\/code> in the <code>SKContext<\/code>.<\/p>\n<pre class=\"prettyprint language-cs language-csharp\"><code class=\"language-cs language-csharp\">\/\/ Search for functions that match the semantic query.\r\nvar memories = context.Memory.SearchAsync(PlannerMemoryCollectionName, semanticQuery, config.MaxRelevantFunctions, config.RelevancyThreshold.Value,\r\n    context.CancellationToken);<\/code><\/pre>\n<p>This is just one basic example of how embeddings and semantic searching can bring LLM capabilities into applications using the Planner. As the Semantic Kernel evolves in its Alpha stage, we&#8217;ll prioritize other methods of using embeddings for plan creation that will be even more powerful.<\/p>\n<h2><strong>Next steps<\/strong><\/h2>\n<p>Read the documentation about the <a href=\"https:\/\/aka.ms\/sk\/concepts\/planner\">Planner<\/a><\/p>\n<p>Join the community and let us know what you think: <a href=\"https:\/\/aka.ms\/sk\/discord\">https:\/\/aka.ms\/sk\/discord<\/a><\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/semantic-kernel\/wp-content\/uploads\/sites\/78\/2023\/03\/skpatternsmallbw.png\" \/><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Semantic Kernel (SK) is a lightweight SDK that lets you mix conventional programming languages, like C# and Python, with the latest in Large Language Model (LLM) AI \u201cprompts\u201d with prompt templating, chaining, and planning capabilities.\u00a0 Its Planner Skill allows users to create and execute plans based on semantic queries. Recently, the addition of embeddings exposed [&hellip;]<\/p>\n","protected":false},"author":115154,"featured_media":5048,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[2,1],"tags":[],"class_list":["post-140","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-samples","category-semantic-kernel"],"acf":[],"blog_post_summary":"<p>Semantic Kernel (SK) is a lightweight SDK that lets you mix conventional programming languages, like C# and Python, with the latest in Large Language Model (LLM) AI \u201cprompts\u201d with prompt templating, chaining, and planning capabilities.\u00a0 Its Planner Skill allows users to create and execute plans based on semantic queries. Recently, the addition of embeddings exposed [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/140","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/users\/115154"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/comments?post=140"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/140\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media\/5048"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media?parent=140"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/categories?post=140"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/tags?post=140"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}