{"id":428,"date":"2023-05-04T14:06:36","date_gmt":"2023-05-04T21:06:36","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/semantic-kernel\/?p=428"},"modified":"2023-09-18T15:02:08","modified_gmt":"2023-09-18T22:02:08","slug":"how-to-use-hugging-face-models-with-semantic-kernel","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/agent-framework\/how-to-use-hugging-face-models-with-semantic-kernel\/","title":{"rendered":"How to use Hugging Face Models with Semantic Kernel"},"content":{"rendered":"

\"Image<\/a><\/h2>\n

We are thrilled to announce the integration of Semantic Kernel with Hugging Face models!<\/p>\n

With this integration, you can leverage the power of Semantic Kernel combined with accessibility of over 190,000+ models from Hugging Face. This integration allows you to use the vast number of models at your fingertips with the latest advancements in Semantic Kernel\u2019s orchestration, skills, planner and contextual memory support.<\/p>\n

What is Hugging Face?<\/h2>\n

Hugging Face<\/a> is a leading provider of open-source models. Models are pre-trained on large datasets and can be used to quickly perform a variety of tasks, such as sentiment analysis, text classification, and text summarization. Using Hugging Face model services can provide great efficiencies as models are pre-trained, easy to swap out and cost-effective with many free models available.<\/p>\n

How to use Semantic Kernel with Hugging Face?<\/h2>\n

This video will give you a walk-through how to get started or dive right into the Python Sample here<\/a>.\u00a0For the remainder of this blog post we will be using the Hugging Face Sample with Skills as reference.<\/p>\n

In the first two cells we install the relevant packages with a pip install and import the Semantic Kernel dependances.<\/p>\n

!python -m pip install -r requirements.txt\r\n\r\nimport semantic_kernel as sk\r\nimport semantic_kernel.connectors.ai.hugging_face as sk_hf<\/code><\/pre>\n
<\/code><\/p>\n
Next, we create a kernel instance and configure the hugging face services we want to use. In this example we will use gp2 for text completion and sentence-transformers\/all-MiniLM-L6-v2 for text embeddings.<\/p>\n
kernel = sk.Kernel()\r\n\r\n# Configure LLM service\r\nkernel.config.add_text_completion_service(\r\n    \"gpt2\", sk_hf.HuggingFaceTextCompletion(\"gpt2\", task=\"text-generation\")\r\n)\r\nkernel.config.add_text_embedding_generation_service(\r\n    \"sentence-transformers\/all-MiniLM-L6-v2\",\r\n    sk_hf.HuggingFaceTextEmbedding(\"sentence-transformers\/all-MiniLM-L6-v2\"),\r\n)\r\nkernel.register_memory_store(memory_store=sk.memory.VolatileMemoryStore())\r\nkernel.import_skill(sk.core_skills.TextMemorySkill())\r\n<\/code><\/pre>\n
We have chosen to use volatile memory, which uses the in-machine memory. We define the text memory skill which we use for this example.<\/p>\n
Now we have Kernel setup, the next cell we define the fact memories we want to the model to reference as it provides us responses. In this example we have facts about animals. Free to edit and get creative as you test this out for yourself. Lastly we create a prompt response template that provides the details on how to respond to our query. That is it! Now we are all set to send our query.<\/p>\n
The last cell in the notebook, defines the query parameters, relevancy and returns our output.<\/p>\n
context = kernel.create_new_context()\r\ncontext[sk.core_skills.TextMemorySkill.COLLECTION_PARAM] = \"animal-facts\"\r\ncontext[sk.core_skills.TextMemorySkill.RELEVANCE_PARAM] = 0.3\r\n\r\ncontext[\"query1\"] = \"animal that swims\"\r\ncontext[\"query2\"] = \"animal that flies\"\r\ncontext[\"query3\"] = \"penguins are?\"\r\noutput = await kernel.run_async(my_function, input_vars=context.variables)\r\n\r\noutput = str(output).strip()\r\n\r\nquery_result1 = await kernel.memory.search_async(\r\n    \"animal-facts\", context[\"query1\"], limit=1, min_relevance_score=0.3\r\n)\r\nquery_result2 = await kernel.memory.search_async(\r\n    \"animal-facts\", context[\"query2\"], limit=1, min_relevance_score=0.3\r\n)\r\nquery_result3 = await kernel.memory.search_async(\r\n    \"animal-facts\", context[\"query3\"], limit=1, min_relevance_score=0.3\r\n)\r\n\r\nprint(f\"gpt2 completed prompt with: '{output}'\")\r\n<\/code><\/pre>\n
Feel free to play with token sizes to vary your response lengths and other parameters to test the different responses.<\/p>\n
Happy testing!<\/p>\n
Next Steps:<\/strong><\/h2>\n
Explore the sample in GitHub<\/a><\/p>\n
Learn more about Semantic Kernel<\/a><\/p>\n
Join the community and let us know what you think:\u00a0https:\/\/aka.ms\/sk\/discord<\/a><\/p>\n
\"Image<\/p>\n","protected":false},"excerpt":{"rendered":"
We are thrilled to announce the integration of Semantic Kernel with Hugging Face models! With this integration, you can leverage the power of Semantic Kernel combined with accessibility of over 190,000+ models from Hugging Face. This integration allows you to use the vast number of models at your fingertips with the latest advancements in Semantic […]<\/p>\n","protected":false},"author":116113,"featured_media":446,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"link","meta":{"_acf_changed":false,"footnotes":""},"categories":[17,2,1],"tags":[],"class_list":["post-428","post","type-post","status-publish","format-link","has-post-thumbnail","hentry","category-announcements","category-samples","category-semantic-kernel","post_format-post-format-link"],"acf":[],"blog_post_summary":"
We are thrilled to announce the integration of Semantic Kernel with Hugging Face models! With this integration, you can leverage the power of Semantic Kernel combined with accessibility of over 190,000+ models from Hugging Face. This integration allows you to use the vast number of models at your fingertips with the latest advancements in Semantic […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/428","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/users\/116113"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/comments?post=428"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/posts\/428\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media\/446"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/media?parent=428"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/categories?post=428"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/agent-framework\/wp-json\/wp\/v2\/tags?post=428"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}