{"id":3341,"date":"2024-09-12T10:58:27","date_gmt":"2024-09-12T17:58:27","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/semantic-kernel\/?p=3341"},"modified":"2025-04-19T06:56:26","modified_gmt":"2025-04-19T13:56:26","slug":"introducing-new-ollama-connector-for-local-models","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/agent-framework\/introducing-new-ollama-connector-for-local-models\/","title":{"rendered":"Introducing new Ollama Connector for Local Models"},"content":{"rendered":"

Introducing new Ollama Connector for Local Models<\/span><\/b>\u00a0<\/span><\/h3>\n

As a long-awaited feature we are happy to bring a dedicated Ollama Connector. In the core implementation of this connector, we are also using the <\/span>Ollama Sharp<\/span><\/a> library which allows you to manipulate some native Ollama APIs as well as using it with our connector for breaking glass scenarios (provide the Ollama Sharp client directly to the AddOllama(Service) extensions).<\/span>\u00a0<\/span><\/p>\n

This connector is currently in alpha pre-release and supports <\/span>Chat Completion<\/span><\/b>, <\/span>Text Embeddings<\/span><\/b> and <\/span>Text Generation<\/span><\/b> capable models.<\/span>\u00a0<\/span><\/p>\n

The biggest benefit of having a dedicated connector for Ollama is that it allows us to support Semantic Kernel features that targeted for Ollama deployed models.<\/span>\u00a0<\/span><\/p>\n

What is Ollama?<\/span><\/b>\u00a0<\/span><\/h3>\n

\"Image<\/a><\/h3>\n

Ollama is an open-source MIT license platform that facilitates the local operation of AI models directly on personal or corporate hardware. It supports a variety of models from different sources, such as Phi-3, Llama-3, Mistral, and many others, allowing users to run these models on their local machines without the need for continuous internet connectivity. This local deployment secures sensitive data and provides complete control over the AI models and their operation.<\/p>\n

The platform is quite simple to use and setup and has been largely adopted by the AI community for edge scenarios and local AI inference tasks.<\/p>\n

Get Started<\/strong><\/h3>\n
    \n
  1. Deploy a model like Phi-3<\/a>. For more details, see the Ollama AI Models library<\/a>.<\/li>\n
  2. Install the Connectors.Ollama<\/a> pre-release package in your existing project from NuGet.<\/li>\n<\/ol>\n

    Add the following code to your application to start making requests to your local AI model. Make sure to provide the correct model Id (phi3, llama3.1, etc.) and the endpoint if different from the default \u201chttp:\/\/localhost:11434\u201d.<\/p>\n

    using<\/span> var<\/span> ollamaClient = new<\/span> OllamaApiClient(\r\n            uriString: \"http:\/\/localhost:11434\",\r\n            defaultModel: \"phi3\");\r\n\r\nvar<\/span> chatService = ollamaClient.AsChatCompletionService();\r\n\r\nvar<\/span> chatHistory = new<\/span> ChatHistory(\"You are a helpful assistant that knows about AI.\"<\/span>); \r\n\r\nchatHistory.AddUserMessage(\"Hi, I'm looking for book suggestions\"<\/span>);\r\n\r\nvar<\/span> reply = await<\/span> chatService.GetChatMessageContentAsync(chatHistory);<\/code><\/pre>\n
    For more scenarios, see the samples below:<\/p>\n