Explore how to integrate GitHub’s AI models, like GPT, Llama and Phi, into your .NET apps using Microsoft’s Semantic Kernel for intelligent applications.

Unlocking the Power of GitHub Models in .NET with Semantic Kernel

The world of AI continues to evolve rapidly, and GitHub has joined the race by introducing a set of popular Large Language Models (LLMs), such as GPT, Llama and Phi, available on the GitHub Marketplace. These models can help developers build powerful AI-driven applications with ease. In this post, we’ll explore how .NET programmers can take advantage of these models and integrate them into their applications using Semantic Kernel.

Introduction to GitHub Models

GitHub has expanded its toolkit by launching GitHub Models, a suite of industry-leading AI Models designed to enable more than 100 million developers to become AI engineers. These models, like Llama 3.1, GPT-4o and Phi-3.5, are particularly helpful for tasks that involve natural language processing (NLP). Available in the GitHub Marketplace, they provide developers a built-in playground that lets them test different prompts and model parameters, for free, right in GitHub.

For .NET developers, these models unlock new possibilities to create intelligent applications that can understand and generate human language or even code, making it easier to streamline various tasks and processes.

Semantic Kernel: A Brief Overview

Semantic Kernel is a lightweight, extensible framework from Microsoft that allows developers to create sophisticated AI applications that leverage LLMs and other cloud services like Azure AI Search. It integrates easily into your .NET applications, making it possible to incorporate natural language understanding and generation features.

With Semantic Kernel, you can define workflows, apply reasoning over the outputs of LLMs, and chain together models to create more complex AI-driven experiences. It acts as a bridge between large language models and your application logic.

Using GitHub Models with Semantic Kernel

To give you a practical example, let’s explore how you can integrate GitHub Models into a C# application using Semantic Kernel. There’s a GitHub repository that provides a working sample of how this integration can be achieved.

Here’s a quick step-by-step guide to get started:

Step 1: Install the necessary NuGet packages

First, ensure you have the required NuGet packages in your C# project:

dotnet add package Microsoft.SemanticKernel --version 1.18.2
dotnet add package Microsoft.Extensions.Configuration.UserSecrets --version 9.0.0-rc.1.24431.7

The Semantic Kernel package allows you to interact with the GitHub Models through the API. Microsoft Configuration User Secrets is used to store and retrieve the required GitHub Token.

Step 2: Setup project secrets with your GitHub Personal Access Token

Generate a new GitHub Personal Access Token. Navigate to the root of your C# project and run these commands to add the Token.

dotnet user-secrets init
dotnet user-secrets set "GH_PAT" "< PAT >"

In the repository Sample console application, these code is used to retrieve:

• GitHub Models, model name
• GitHub Models, model endpoint
• GitHub Personal Access Token

var config = new ConfigurationBuilder().AddUserSecrets<Program>().Build();
var modelId = "Phi-3.5-mini-instruct";
var uri = "https://models.inference.ai.azure.com";
var githubPAT = config["GH_PAT"];

This is an example of how to set the modelId and the uri, and the GitHub PAT using secrets:

Step 3: Configure the Semantic Kernel client to use GitHub Models

Next, set up the Semantic Kernel to integrate with the GitHub models API:

// create client
var client = new OpenAIClient(new ApiKeyCredential(githubPAT), new OpenAIClientOptions { Endpoint = new Uri(uri) });

// Create a chat completion service
var builder = Kernel.CreateBuilder();
builder.AddOpenAIChatCompletion(modelId, client);

// Get the chat completion service
Kernel kernel = builder.Build();
var chat = kernel.GetRequiredService<IChatCompletionService>();

Step 4: Run the App

Now, define the task you want the GitHub model to perform. The sample console app, is a standard Q&A chat that runs in the console:

var history = new ChatHistory();
history.AddSystemMessage("You are a useful chatbot. If you don't know an answer, say 'I don't know!'. Always reply in a funny way. Use emojis if possible.");

while (true)
{
    Console.Write("Q: ");
    var userQ = Console.ReadLine();
    if (string.IsNullOrEmpty(userQ))
    {
        break;
    }
    history.AddUserMessage(userQ);

    var sb = new StringBuilder();
    var result = chat.GetStreamingChatMessageContentsAsync(history);
    Console.Write("AI: ");
    await foreach (var item in result)
    {
        sb.Append(item);
        Console.Write(item.Content);
    }
    Console.WriteLine();

    history.AddAssistantMessage(sb.ToString());
}

Optional: The repo is ready to run the sample project using Codespaces. The chat demo application should look like these:

Summary

Integrating GitHub Models into your .NET applications using Semantic Kernel opens up exciting possibilities for building AI-driven applications. With tools like Semantic Kernel, you can streamline your development process and create smarter applications.

If you’re looking to dive deeper into this topic, check out the following resources:

• Introducing GitHub Models
• GitHub Repository with Semantic Kernel Integration

Happy coding!