{"id":56759,"date":"2025-05-15T10:05:00","date_gmt":"2025-05-15T17:05:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/dotnet\/?p=56759"},"modified":"2025-05-15T10:05:00","modified_gmt":"2025-05-15T17:05:00","slug":"evaluating-ai-content-safety","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/evaluating-ai-content-safety\/","title":{"rendered":"Evaluating content safety in your .NET AI applications"},"content":{"rendered":"<p>We are excited to announce the addition of the <a href=\"https:\/\/www.nuget.org\/packages\/Microsoft.Extensions.AI.Evaluation.Safety\">Microsoft.Extensions.AI.Evaluation.Safety<\/a> package to the Microsoft.Extensions.AI.Evaluation libraries! This new package provides evaluators that help you detect harmful or sensitive content \u2014 such as hate speech, violence, copyrighted material, insecure code, and <a href=\"https:\/\/learn.microsoft.com\/dotnet\/ai\/conceptual\/evaluation-libraries#safety-evaluators\">more<\/a> \u2014 within AI-generated content in your Intelligent Applications.<\/p>\n<p>These safety evaluators are powered by the <a href=\"https:\/\/learn.microsoft.com\/azure\/ai-foundry\/concepts\/evaluation-metrics-built-in\">Azure AI Foundry Evaluation service<\/a> and are designed for seamless integration into your existing workflows, whether you&#8217;re running evaluations within unit tests locally or automating offline evaluation checks in your CI\/CD pipelines.<\/p>\n<p>The new <a href=\"https:\/\/learn.microsoft.com\/dotnet\/ai\/conceptual\/evaluation-libraries#safety-evaluators\">safety evaluators<\/a> complement the <a href=\"https:\/\/learn.microsoft.com\/dotnet\/ai\/conceptual\/evaluation-libraries#quality-evaluators\">quality-focused evaluators<\/a> we covered earlier in the below posts. Together, they provide a comprehensive toolkit for evaluating AI-generated content in your applications.<\/p>\n<ul>\n<li><a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/evaluate-the-quality-of-your-ai-applications-with-ease\/\">Evaluate the quality of your AI applications with ease<\/a><\/li>\n<li><a href=\"https:\/\/devblogs.microsoft.com\/dotnet\/start-using-the-microsoft-ai-evaluations-library-today\/\">Unlock new possibilities for AI Evaluations for .NET<\/a><\/li>\n<\/ul>\n<h2>Setting Up Azure AI Foundry for Safety Evaluations<\/h2>\n<p>To use the safety evaluators, you&#8217;ll need the following steps to set up access to the Azure AI Foundry Evaluation service:<\/p>\n<ol>\n<li>Firstly, you need an <a href=\"https:\/\/azure.microsoft.com\/\">Azure subscription<\/a>.<\/li>\n<li>Within this subscription, create a <a href=\"https:\/\/learn.microsoft.com\/azure\/azure-resource-manager\/management\/manage-resource-groups-portal\">resource group<\/a> within one of the <a href=\"https:\/\/learn.microsoft.com\/azure\/ai-foundry\/how-to\/develop\/evaluate-sdk#region-support\">Azure regions that support Azure AI Foundry Evaluation service<\/a>.<\/li>\n<li>Next create an <a href=\"https:\/\/learn.microsoft.com\/azure\/ai-foundry\/concepts\/ai-resources\">Azure AI hub<\/a> within the same resource group and region.<\/li>\n<li>Finally, create an <a href=\"https:\/\/learn.microsoft.com\/azure\/ai-foundry\/how-to\/create-projects?tabs=ai-studio\">Azure AI project<\/a> within this hub.<\/li>\n<li>Once you have created the above artifacts, configure the following environment variables so that the evaluators used in the code example below can connect to the above Azure AI project:<\/li>\n<\/ol>\n<pre><code class=\"language-sh\">set EVAL_SAMPLE_AZURE_SUBSCRIPTION_ID=&lt;your-subscription-id&gt;\nset EVAL_SAMPLE_AZURE_RESOURCE_GROUP=&lt;your-resource-group-name&gt;\nset EVAL_SAMPLE_AZURE_AI_PROJECT=&lt;your-ai-project-name&gt;<\/code><\/pre>\n<h2>C# Example: Evaluating Content Safety<\/h2>\n<p>The following code shows how to configure and run safety evaluators to check an AI response for violence, hate and unfairness, protected material, and indirect attacks.<\/p>\n<p>To run this example, create a new MSTest unit test project. Make sure to do this from a command prompt or terminal where the above environment variables (<code>EVAL_SAMPLE_AZURE_SUBSCRIPTION_ID<\/code>, <code>EVAL_SAMPLE_AZURE_RESOURCE_GROUP<\/code>, <code>EVAL_SAMPLE_AZURE_AI_PROJECT<\/code>) are already set.<\/p>\n<p>You can create the project using either Visual Studio or the .NET CLI:<\/p>\n<p><strong>Using Visual Studio:<\/strong><\/p>\n<ol>\n<li>Open Visual Studio.<\/li>\n<li>Select <strong>File &gt; New &gt; Project&#8230;<\/strong><\/li>\n<li>Search for and select <strong>MSTest Test Project<\/strong>.<\/li>\n<li>Choose a name and location, then click <strong>Create<\/strong>.<\/li>\n<\/ol>\n<p><strong>Using Visual Studio Code with C# Dev Kit:<\/strong><\/p>\n<ol>\n<li>Open Visual Studio Code.<\/li>\n<li>Open Command Pallet and select <strong>.NET: New Project&#8230;<\/strong><\/li>\n<li>Select <strong>MSTest Test Project<\/strong>.<\/li>\n<li>Choose a name and location, then select <strong>Create Project<\/strong>.<\/li>\n<\/ol>\n<p><strong>Using the .NET CLI:<\/strong><\/p>\n<pre><code class=\"language-sh\">dotnet new mstest -n SafetyEvaluationTests\ncd SafetyEvaluationTests<\/code><\/pre>\n<p>After creating the project, add the necessary NuGet packages:<\/p>\n<pre><code class=\"language-sh\">dotnet add package Microsoft.Extensions.AI.Evaluation --prerelease\ndotnet add package Microsoft.Extensions.AI.Evaluation.Safety --prerelease\ndotnet add package Microsoft.Extensions.AI.Evaluation.Reporting --prerelease\ndotnet add package Azure.Identity<\/code><\/pre>\n<p>Then, copy the following code into the project (inside Test1.cs).<\/p>\n<pre><code class=\"language-csharp\">using Azure.Identity;\nusing Microsoft.Extensions.AI.Evaluation;\nusing Microsoft.Extensions.AI.Evaluation.Reporting;\nusing Microsoft.Extensions.AI.Evaluation.Reporting.Storage;\nusing Microsoft.Extensions.AI.Evaluation.Safety;\n\nnamespace SafetyEvaluationTests;\n\n[TestClass]\npublic class Test1\n{\n    [TestMethod]\n    public async Task EvaluateContentSafety()\n    {\n        \/\/ Configure the Azure AI Foundry Evaluation service.\n        var contentSafetyServiceConfig = \n            new ContentSafetyServiceConfiguration(\n                credential: new DefaultAzureCredential(),\n                subscriptionId: Environment.GetEnvironmentVariable(\"EVAL_SAMPLE_AZURE_SUBSCRIPTION_ID\")!,\n                resourceGroupName: Environment.GetEnvironmentVariable(\"EVAL_SAMPLE_AZURE_RESOURCE_GROUP\")!,\n                projectName: Environment.GetEnvironmentVariable(\"EVAL_SAMPLE_AZURE_AI_PROJECT\")!);\n\n        \/\/ Create a reporting configuration with the desired content safety evaluators.\n        \/\/ The evaluation results will be persisted to disk under the storageRootPath specified below.\n        ReportingConfiguration reportingConfig = DiskBasedReportingConfiguration.Create(\n            storageRootPath: \".\/eval-results\",\n            evaluators: new IEvaluator[]\n            {\n                new ViolenceEvaluator(),\n                new HateAndUnfairnessEvaluator(),\n                new ProtectedMaterialEvaluator(),\n                new IndirectAttackEvaluator()\n            },\n            chatConfiguration: contentSafetyServiceConfig.ToChatConfiguration(),\n            enableResponseCaching: true);\n\n        \/\/ Since response caching is enabled above, the responses from the Azure AI Foundry Evaluation service will\n        \/\/ also be cached under the storageRootPath so long as the response being evaluated (below) stays unchanged,\n        \/\/ and so long as the cache entry does not expire (cache expiry is set at 14 days by default).\n\n        \/\/ Define the AI request and response to be evaluated. The response is hard coded below for ease of\n        \/\/ demonstration. But you can also fetch the response from an LLM.\n        string query = \"How far is the Sun from the Earth at its closest and furthest points?\";\n        string response =\n            \"\"\"\n            The distance between the Sun and Earth isn\u2019t constant.\n            It changes because Earth's orbit is elliptical rather than a perfect circle.\n            At its closest point (Perihelion): About 147 million kilometers (91 million miles).\n            At its furthest point (Aphelion): Roughly 152 million kilometers (94 million miles).\n            \"\"\";\n\n        \/\/ Run the evaluation.\n        await using ScenarioRun scenarioRun =\n            await reportingConfig.CreateScenarioRunAsync(\"Content Safety Evaluation Example\");\n\n        EvaluationResult result = await scenarioRun.EvaluateAsync(query, response);\n\n        \/\/ Retrieve one of the metrics (example: Violence).\n        NumericMetric violence = result.Get&lt;NumericMetric&gt;(ViolenceEvaluator.ViolenceMetricName);\n        Assert.IsFalse(violence.Interpretation!.Failed);\n        Assert.IsTrue(violence.Value &lt; 2);\n    }\n}<\/code><\/pre>\n<h2>Running the Example and Generating Reports<\/h2>\n<p>Next, let&#8217;s run the above unit test. You can either use Visual Studio or Visual Studio Code&#8217;s Test Explorer or run <code>dotnet test<\/code> from the command line.<\/p>\n<p>After running the test, you can generate an HTML report of the evaluated metrics using the <a href=\"https:\/\/www.nuget.org\/packages\/Microsoft.Extensions.AI.Evaluation.Console\"><code>dotnet aieval<\/code><\/a> tool. Install the tool locally under the project folder by running:<\/p>\n<pre><code class=\"language-sh\">dotnet tool install Microsoft.Extensions.AI.Evaluation.Console --create-manifest-if-needed --prerelease<\/code><\/pre>\n<p>Then generate and view the report:<\/p>\n<pre><code class=\"language-sh\">dotnet aieval report -p &lt;path to 'eval-results' folder under the build output directory for the above project&gt; -o .\/report.html --open<\/code><\/pre>\n<p>Here&#8217;s a peek at the generated report &#8211; the report is interactive, and this screenshot shows the details revealed when you click on the Indirect Attack metric.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2025\/05\/report.png\" alt=\"The Evaluation Report\" \/><\/p>\n<h2>More Samples<\/h2>\n<p>The <a href=\"https:\/\/github.com\/dotnet\/ai-samples\/tree\/main\/src\/microsoft-extensions-ai-evaluation\">API usage samples<\/a> for the libraries demonstrate several additional scenarios, including:<\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/dotnet\/ai-samples\/blob\/main\/src\/microsoft-extensions-ai-evaluation\/api\/reporting\/ReportingExamples.Example09_RunningSafetyEvaluatorsAgainstResponsesWithImages.cs\">Evaluating content safety of AI responses containing images<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/dotnet\/ai-samples\/blob\/main\/src\/microsoft-extensions-ai-evaluation\/api\/reporting\/ReportingExamples.Example10_RunningQualityAndSafetyEvaluatorsTogether.cs\">Running safety and quality evaluators together<\/a><\/li>\n<\/ul>\n<p>These samples also contain examples that provide guidance on best practices, such as <a href=\"https:\/\/github.com\/dotnet\/ai-samples\/blob\/main\/src\/microsoft-extensions-ai-evaluation\/api\/reporting\/ReportingExamples.cs#L92\">sharing evaluator and reporting configurations across multiple tests<\/a>, setting up result storage, execution names and response caching, <a href=\"https:\/\/github.com\/dotnet\/ai-samples\/blob\/main\/src\/microsoft-extensions-ai-evaluation\/api\/INSTRUCTIONS.md#generating-reports-using-the-aieval-dotnet-tool\">installing and running the <code>aieval<\/code> tool and using it as part of your CI\/CD pipelines<\/a> etc. If you haven&#8217;t run these samples before, be sure to check out the included <a href=\"https:\/\/github.com\/dotnet\/ai-samples\/blob\/main\/src\/microsoft-extensions-ai-evaluation\/api\/INSTRUCTIONS.md\">instructions<\/a> first.<\/p>\n<h2>Other Updates<\/h2>\n<p>In addition to launching the new content safety evaluators, we&#8217;ve been hard at work enhancing the Microsoft.Extensions.AI.Evaluation libraries with even more powerful features.<\/p>\n<ul>\n<li>\n<p>The <a href=\"https:\/\/www.nuget.org\/packages\/Microsoft.Extensions.AI.Evaluation.Quality\">Microsoft.Extensions.AI.Evaluation.Quality<\/a> package now offers an expanded suite of evaluators, including the recently added <a href=\"https:\/\/learn.microsoft.com\/dotnet\/api\/microsoft.extensions.ai.evaluation.quality.retrievalevaluator\">RetrievalEvaluator<\/a>, <a href=\"https:\/\/learn.microsoft.com\/dotnet\/api\/microsoft.extensions.ai.evaluation.quality.relevanceevaluator\">RelevanceEvaluator<\/a>, and <a href=\"https:\/\/learn.microsoft.com\/dotnet\/api\/microsoft.extensions.ai.evaluation.quality.completenessevaluator\">CompletenessEvaluator<\/a>. Whether you&#8217;re measuring how well your AI system retrieves information, stays on topic, or delivers complete answers, there&#8217;s an evaluator ready for the job. <a href=\"https:\/\/learn.microsoft.com\/dotnet\/ai\/conceptual\/evaluation-libraries#quality-evaluators\">Explore the full list of available evaluators<\/a>.<\/p>\n<\/li>\n<li>\n<p>The reporting functionality has also seen significant upgrades, making it easier than ever to gain insights from your evaluation runs. You can now:<\/p>\n<ul>\n<li>Search and filter scenarios using tags for faster navigation.<\/li>\n<li>View rich metadata and diagnostics for each metric \u2014 including details like token usage and latency.<\/li>\n<li>Track historical trends for every metric, visualizing score changes and pass\/fail rates across multiple executions right within the scenario tree.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h2>Get Started Today<\/h2>\n<p>Ready to take your AI application&#8217;s quality and safety to the next level? Dive into the Microsoft.Extensions.AI.Evaluation libraries and experiment with the powerful new content safety evaluators in the Microsoft.Extensions.AI.Evaluation.Safety package. We can&#8217;t wait to see the innovative ways you&#8217;ll put these tools to work!<\/p>\n<p>Stay tuned for future enhancements and updates. We encourage you to share your <a href=\"https:\/\/github.com\/dotnet\/extensions\/issues?q=is%3Aissue%20state%3Aopen%20label%3Aarea-ai-eval\">feedback and contributions<\/a> to help us continue improving these libraries. Happy evaluating!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Announcing content safety evaluations and other imporvements in the Microsoft.Extensions.AI.Evaluation libraries.<\/p>\n","protected":false},"author":175748,"featured_media":56760,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[685,7781,756],"tags":[4,568,46,7903,7870,8040,7885,8041,136],"class_list":["post-56759","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet","category-ai","category-csharp","tag-net","tag-ai","tag-c","tag-evaluations","tag-meai","tag-meai-evaluation","tag-microsoft-extensions-ai","tag-microsoft-extensions-ai-evaluation","tag-testing"],"acf":[],"blog_post_summary":"<p>Announcing content safety evaluations and other imporvements in the Microsoft.Extensions.AI.Evaluation libraries.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/56759","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/175748"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=56759"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/56759\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/56760"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=56759"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=56759"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=56759"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}