Today we’re featuring a guest author, Ricardo Niepel, who’s a Senior Technical Specialist at Microsoft. He’s written an article we’re sharing below; we’ll turn it over to Ricardo to share more!

If “Creative Writing Assistant” sounds familiar, it might be because we’ve previously released a similar multi-agent application built in Python during our Ignite conference (Azure AI Foundry and the dev toolchain to infuse AI in all your apps | Ignite Session BRK119). Based on the invaluable feedback we received from all of you we are now excited to show a .NET version of this sample application. It illustrates how the .NET ecosystem, coupled with Semantic Kernel, can be harnessed to create robust multi-agent applications in the enterprise.

Introducing the C# version of Creative Writing Assistant

The Creative Writing Assistant is an app that will help you write well researched, product specific articles. Enter the required information and then click “Send”. Inside the chat you can see the different agents working on the final article and what happens underneath. At the end the result will be acknowledged by the editor agent and the input field enables you to send another input into it.

This sample consists of four different agents collaborating in writing the article:

• Research Agent This agent uses your context / question, formulates out of that expert queries and uses Bing search to get results for them. After that the findings are summarized.
• Marketing Agent This agent uses your context / question, formulates out of that up to 5 specialized queries and uses an internal Vector store to find matching products with semantic search. After that the findings are summarized.
• Writer Agent This agent uses your instructions, the other contexts and the research and marketing results to produce an article. If it gets feedback for rework, it will do so.
• Editor Agent Reviews the outcomes from the writer agent and provides feedback to it. This agent also decides when the article is accepted and no further rework necessary.

Technology wise this sample app is written in C# and the frontend in TypeScript, where .NET Aspire is used as the underlying framework to bundle such kind of distributed applications together. For the orchestration of the different agents, the integration of external tools, like Bing Search API, or the usage of Azure AI Search as vector store to enable a semantic similarity search for related products, Semantic Kernel and more specifically the Semantic Kernel Agent Framework, currently in preview, is used. As a standard protocol for developing chat interfaces the @microsoft/ai-chat-protocol spec and corresponding SDK is leveraged.

Using Polyglot Notebooks as Ideating and Exploring loop

This first loop in the lifecycle of building enterprise generative AI applications typically involves a smaller round of developers searching for a model, working only with a subset of data and prompts, refining the prompts, and try to find an promising approach to tackle the business challenge.

As multi-agent applications heavily rely on software, integration points, tools and different agent communication patterns, it is not enough to test isolated prompts with the completion APIs. This sample app uses Polyglot Notebooks in Visual Studio Code to create interactive prototyping environments using C# with Semantic Kernel.

Two different notebooks are included, to show the evolution from a simple WriterReviewer agent scenario to the full end-result of having four different agents collaborating and acting as the CreativeWritingAssistant.

Leveraging Microsoft.Extensions.AI.Evaluation

This sample app includes automated tests to evaluate the application response quality which is based on generative AI. The 4 metrics the evaluators in this project assess are Coherence, Fluency, Relevance and Groundedness.

The Microsoft.Extensions.AI.Evaluation library, currently in preview, is used to execute the evaluators, cache the LLM responses, store the results of evaluations and generating reports from that data.

To simulate an end-to-end test, sample scenarios are sent to the API so that each scenario will have the evaluations run and will incorporate all the context, research, products, and final article when grading the response. The test infrastructure setup is using Aspire.Hosting.Testing to create a test host for distributed applications.

Operationalization

As .NET Aspire and also Semantic Kernel have built-in OpenTelemetry integrations, this sample leverages these to be able to observe the behavior of the different services and agents. Semantic Kernel adheres to OpenTelemetry Semantic Convention, ensuring structured logs, metrics, and traces for effective analysis. The Semantic Conventions for Generative AI are currently experimental, but Semantic Kernel strives to follow the OpenTelemetry Semantic Convention as closely as possible to provide a meaningful observability experience for AI solutions.

The sample application uses the .NET Aspire integration with the Azure Developer CLI (azd) to enable fully automated deployments both in the developer inner loop and in CICD processes. As target runtime Azure Container Apps is used for the application itself. Everything is configured in a way that identities are used, and connection strings are eliminated to the bare minimum. In this situation only Bing Search needs a secret anymore, which is then stored inside Key Vault, handled completely by .NET Aspire.

Expandable in the future

To ensure the sample app remains a living and evolving entity, it will continually receive updates as the Semantic Kernel Agent framework matures. This dynamic approach will allow the application to integrate the latest advancements and improvements, thereby maintaining its relevance and effectiveness. By incorporating the newest features and best practices, we aim to provide a robust and cutting-edge sample that adapts to the ever-changing landscape of building LLM applications AI and software development.

Once supported within Semantic Kernel, we will integrate the Azure AI Agent Service into it, to show how to leverage a fully managed service having extensible AI agents without needing to manage the underlying compute and storage resources. This will simplify the setup of the Bing Search and Azure AI Search integration, as we could leverage the built-in functionality to achieve the same outcomes.

We are also exploring incorporating human input into the review process by adding another AgentChat participant, such as a UserProxyAgent. This approach will show how to integrate humans in the loop when building AI applications and leveraging Semantic Kernel.

If you have other ideas or feedback, how this sample application can be further improved or expanded, please reach out to us directly on GitHub.

Conclusion

In conclusion, combining the Semantic Kernel Agent framework with .NET Aspire demonstrates how to leverage C# in building robust, enterprise-ready, multi-agent applications. We invite you to engage with us, share your ideas, and contribute to this project’s evolution.

As next steps you can clone and run it locally by following the instructions here: https://github.com/Azure-Samples/aspire-semantic-kernel-creative-writer?tab=readme-ov-file#getting-started

Please reach out if you have any questions or feedback through our Semantic Kernel GitHub Discussion Channel. We look forward to hearing from you!