{"id":60507,"date":"2026-07-31T08:00:15","date_gmt":"2026-07-31T15:00:15","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/dotnet\/?p=60507"},"modified":"2026-07-30T15:59:41","modified_gmt":"2026-07-30T22:59:41","slug":"polyglot-unit-testing-agent","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/polyglot-unit-testing-agent\/","title":{"rendered":"From generated code to trusted code with a unit-test agent"},"content":{"rendered":"<p>A common request to a coding agent is only one line:<\/p>\n<blockquote><p>Generate unit tests.<\/p><\/blockquote>\n<p>That request leaves important questions open. Which code needs tests? Which\ntest framework does the project use? Where should the tests go? How does the\nbuild find them? What should the tests check?<\/p>\n<p>The agent can learn from the repository, write the tests, and prove that they\nwork.<\/p>\n<p>This is why we built\n<a href=\"https:\/\/github.com\/dotnet\/skills\/blob\/main\/plugins\/dotnet-test\/agents\/code-testing-generator.agent.md\"><code>code-testing-generator<\/code><\/a>.\nIt is an open-source, polyglot agent for unit-test generation. You can find it\nin the\n<a href=\"https:\/\/github.com\/dotnet\/skills\/tree\/main\/plugins\/dotnet-test\"><code>dotnet-test<\/code> plugin<\/a>\nin <a href=\"https:\/\/github.com\/dotnet\/skills\"><code>dotnet\/skills<\/code><\/a>.<\/p>\n<p>The agent writes <strong>unit tests<\/strong>. It isolates the code under test and mocks\nexternal services and other outside dependencies.<\/p>\n<p>Integration, end-to-end, browser, and performance tests are outside its current\nscope.<\/p>\n<p>It does more than make the tests pass. It checks the assertions, the requested\nscenarios, and whether the repository&#8217;s normal test run finds the new tests.<\/p>\n<h2>What happens after the prompt<\/h2>\n<p>The agent does not start writing tests immediately. It learns from the\nrepository first, then plans, writes, and checks the tests.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2026\/07\/unit-testing-agent-workflow.png\" alt=\"Workflow from repository research through planning, implementation, test execution, and quality checks\" \/><\/p>\n<h3>First, learn from the repository<\/h3>\n<p>The agent searches the repository for the code that needs tests. It detects the\nlanguage and test framework. It also looks for existing tests that show where\nnew tests belong and how they should look.<\/p>\n<p>It also finds the correct commands to build and run the tests.<\/p>\n<p>This prevents a common problem. A new test project can build and pass on its\nown but never run in continuous integration (CI) because it was not added to\nthe solution or the repository&#8217;s test command. The agent checks how the\nrepository discovers tests and confirms that the new tests appear there.<\/p>\n<h3>Next, choose the right amount of work<\/h3>\n<p>A single method does not need a large plan. A whole solution does.<\/p>\n<p>The agent chooses from three paths:<\/p>\n<ul>\n<li><strong>Direct:<\/strong> Read the relevant code, write the tests, and validate the result.<\/li>\n<li><strong>Single pass:<\/strong> Research and plan once, then implement that plan.<\/li>\n<li><strong>Iterative:<\/strong> Repeat the cycle to cover a large request or reach a coverage\ngoal.<\/li>\n<\/ul>\n<h3>Then, plan and write the tests<\/h3>\n<p>For larger tasks, the agent lists the code that needs tests. It starts with\nsimple code and then moves to code with more dependencies. It maps each\nbehavior to a test file.<\/p>\n<p>The plan follows the request. A request for one module should not change every\ntest project in the repository.<\/p>\n<p>The agent follows the local conventions and runs the tests as it works. If the\ngenerated code does not compile, it fixes it. If an assertion is wrong, it\nreads the source and corrects the test.<\/p>\n<p>It does not change production code during test generation. It also avoids unit\ntests that call external URLs, open ports, or depend on exact timing.<\/p>\n<h3>Finally, check that the tests are useful<\/h3>\n<p>A test can pass but provide little or no added value. For example, it may only\ncheck that a result is not null. It may test the wrong method. It may even pass\nif the method always returns a default value.<\/p>\n<p>The agent checks for these problems before it finishes:<\/p>\n<ol>\n<li>It considers small code changes that should make the tests fail. This is a\nlightweight form of mutation testing.<\/li>\n<li>It looks for weak or missing assertions.<\/li>\n<li>It checks that every requested scenario has a matching test.<\/li>\n<li>It builds the full workspace and runs the full test suite.<\/li>\n<li>It confirms that the repository&#8217;s test command can find the new tests.<\/li>\n<\/ol>\n<p>This workflow turns a short prompt into a tested result.<\/p>\n<h2>What we measured<\/h2>\n<p>In our latest benchmark, the agent completed 140 of 152 tasks. Stock Copilot,\nwhich used the same tool without our plugin, completed 120 with the same model.\nThat is a <strong>92.1% completion rate versus 78.9%<\/strong>, with 63% fewer failures.<\/p>\n<p>The biggest gains came from prompts like the one at the start of this post. We\ncall these <strong>vague prompts<\/strong> because they leave most decisions to the agent.<\/p>\n<h3>Vague prompts show why the workflow matters<\/h3>\n<p>Vague prompts produced the clearest result. The agent passed 79 of 89 tasks,\ncompared with 59 for stock Copilot. Failures fell from 30 to 10.<\/p>\n<p>We used our internal unit-testing benchmark. It contains 152 tasks from real\nrepositories. Some prompts are detailed. Others are vague and leave most\ndecisions to the agent.<\/p>\n<p>We compared four setups: stock GitHub Copilot, stock Claude Code, our agent in\nGitHub Copilot, and our agent in Claude Code. The results below focus on the\nGitHub Copilot comparison unless we say otherwise. Both Copilot setups used the\nsame model and prompts.<\/p>\n<p>A task passed only when:<\/p>\n<ul>\n<li>The repository built.<\/li>\n<li>All tests passed.<\/li>\n<li>The agent added at least one test.<\/li>\n<li>The agent did not remove any existing tests.<\/li>\n<\/ul>\n<p>The full breakdown shows that detailed prompts were tied:<\/p>\n<table>\n<thead>\n<tr>\n<th>Prompt type<\/th>\n<th style=\"text-align: right;\">Specialized agent<\/th>\n<th style=\"text-align: right;\">Stock Copilot<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Vague prompts, 89 tasks<\/td>\n<td style=\"text-align: right;\">79 (88.8%)<\/td>\n<td style=\"text-align: right;\">59 (66.3%)<\/td>\n<\/tr>\n<tr>\n<td>Detailed prompts, 63 tasks<\/td>\n<td style=\"text-align: right;\">61 (96.8%)<\/td>\n<td style=\"text-align: right;\">61 (96.8%)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2026\/07\/vague-prompt-results.png\" alt=\"Specialized resolved 88.8% of vague prompts versus 66.3% for stock; both resolved 96.8% of detailed prompts\" \/><\/p>\n<p>That is <strong>67% fewer failures<\/strong> on vague prompts. All 20 net gains came from\nthese tasks. This matches the goal of the agent: do the research that the\ndeveloper did not put in the prompt.<\/p>\n<p>The same pattern appeared when the prompt pointed to a code change. The\nbenchmark has 15 tasks that ask for tests for a specific diff. The agent passed\nall 15. Stock Copilot passed none.<\/p>\n<h3>More successful results, not simply more tests<\/h3>\n<p>Both setups ran the same 152 tasks. This lets us compare each result directly.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2026\/07\/paired-benchmark-outcomes.png\" alt=\"Paired outcomes: both resolved 119 tasks, specialized only 21, stock only one, and neither 11\" \/><\/p>\n<ul>\n<li>Both passed 119 tasks.<\/li>\n<li>Only the specialized agent passed 21 tasks.<\/li>\n<li>Only stock Copilot passed one task.<\/li>\n<li>Both failed 11 tasks.<\/li>\n<\/ul>\n<table>\n<thead>\n<tr>\n<th>Metric<\/th>\n<th style=\"text-align: right;\">Specialized agent<\/th>\n<th style=\"text-align: right;\">Stock Copilot<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Tasks completed<\/td>\n<td style=\"text-align: right;\">140\/152 (92.1%)<\/td>\n<td style=\"text-align: right;\">120\/152 (78.9%)<\/td>\n<\/tr>\n<tr>\n<td>Solutions building<\/td>\n<td style=\"text-align: right;\">148<\/td>\n<td style=\"text-align: right;\">145<\/td>\n<\/tr>\n<tr>\n<td>Final test suites passing<\/td>\n<td style=\"text-align: right;\">149<\/td>\n<td style=\"text-align: right;\">147<\/td>\n<\/tr>\n<tr>\n<td>Tests generated<\/td>\n<td style=\"text-align: right;\">6,963<\/td>\n<td style=\"text-align: right;\">7,129<\/td>\n<\/tr>\n<tr>\n<td>Average final line coverage<\/td>\n<td style=\"text-align: right;\">72.4%<\/td>\n<td style=\"text-align: right;\">72.2%<\/td>\n<\/tr>\n<tr>\n<td>Average final branch coverage<\/td>\n<td style=\"text-align: right;\">49.8%<\/td>\n<td style=\"text-align: right;\">49.1%<\/td>\n<\/tr>\n<tr>\n<td>Average task time<\/td>\n<td style=\"text-align: right;\">359 seconds<\/td>\n<td style=\"text-align: right;\">380 seconds<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The specialized agent generated 2.3% fewer tests, with effectively the same\naverage coverage. It also completed more tasks and was about 5.5% faster on\naverage. The gain came from reliability, not from producing more tests.<\/p>\n<h3>The benefit extends across models and languages<\/h3>\n<p>The benchmark contains 45 .NET tasks. They cover several repositories and\ndifferent task sizes. We ran the same comparison with three models:<\/p>\n<table>\n<thead>\n<tr>\n<th>Model<\/th>\n<th style=\"text-align: right;\">Specialized agent<\/th>\n<th style=\"text-align: right;\">Stock Copilot<\/th>\n<th style=\"text-align: right;\">Failure reduction<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Claude Opus 4.8<\/td>\n<td style=\"text-align: right;\">43\/45 (95.6%)<\/td>\n<td style=\"text-align: right;\">35\/45 (77.8%)<\/td>\n<td style=\"text-align: right;\">80%<\/td>\n<\/tr>\n<tr>\n<td>GPT-5.5<\/td>\n<td style=\"text-align: right;\">41\/45 (91.1%)<\/td>\n<td style=\"text-align: right;\">36\/45 (80.0%)<\/td>\n<td style=\"text-align: right;\">56%<\/td>\n<\/tr>\n<tr>\n<td>Claude Haiku 4.5<\/td>\n<td style=\"text-align: right;\">34\/45 (75.6%)<\/td>\n<td style=\"text-align: right;\">25\/45 (55.6%)<\/td>\n<td style=\"text-align: right;\">45%<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/dotnet\/wp-content\/uploads\/sites\/10\/2026\/07\/csharp-failures-by-model.png\" alt=\"C# unresolved tasks: specialized versus stock was 2 to 10 on Opus, 4 to 9 on GPT-5.5, and 11 to 20 on Haiku\" \/><\/p>\n<p>The workflow helped every model. With Opus, it added eight wins with no losses.\nWith Haiku, it completed nine more C# tasks than stock Haiku.<\/p>\n<p>The result also suggests that a strong workflow can lift a mid-tier model close\nto the best result. Across all 152 tasks, specialized GPT-5.5 reached 90.1%.\nThat was within two points of specialized Opus and more than 11 points above\nstock Opus.<\/p>\n<p>The agent includes guidance for .NET, Python, TypeScript, JavaScript, Java, Go,\nRuby, Rust, Swift, Kotlin, PowerShell, and C++. It learns each repository&#8217;s\nconventions instead of applying C# patterns everywhere.<\/p>\n<p>Beyond .NET, the same Opus run covered several of these languages:<\/p>\n<table>\n<thead>\n<tr>\n<th>Language<\/th>\n<th style=\"text-align: right;\">Specialized agent<\/th>\n<th style=\"text-align: right;\">Stock Copilot<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Python, 15 tasks<\/td>\n<td style=\"text-align: right;\">13 (86.7%)<\/td>\n<td style=\"text-align: right;\">6 (40.0%)<\/td>\n<\/tr>\n<tr>\n<td>Go, 15 tasks<\/td>\n<td style=\"text-align: right;\">15 (100%)<\/td>\n<td style=\"text-align: right;\">10 (66.7%)<\/td>\n<\/tr>\n<tr>\n<td>PowerShell, 10 tasks<\/td>\n<td style=\"text-align: right;\">7 (70.0%)<\/td>\n<td style=\"text-align: right;\">8 (80.0%)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The agent passed every Go task and more than doubled the Python completion\nrate. It also passed five Go and five Python tasks that targeted a specific\ncode change.<\/p>\n<p>The result was not better in every language. Stock Copilot passed one more\nPowerShell task. These groups are also small, so we see them as useful signals,\nnot promises for every repository.<\/p>\n<h3>A second, harder benchmark<\/h3>\n<p>We also tested the workflow on 44 unit-test tasks from\n<a href=\"https:\/\/github.com\/scaleapi\/SWE-Atlas\">SWE Atlas<\/a>. This benchmark checks\nrequirements and uses code changes to see whether the tests can catch bugs.<\/p>\n<table>\n<thead>\n<tr>\n<th>Metric<\/th>\n<th style=\"text-align: right;\">Specialized agent<\/th>\n<th style=\"text-align: right;\">Stock Copilot<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Tasks completed<\/td>\n<td style=\"text-align: right;\">16\/44 (36.4%)<\/td>\n<td style=\"text-align: right;\">12\/44 (27.3%)<\/td>\n<\/tr>\n<tr>\n<td>Wins by only one setup<\/td>\n<td style=\"text-align: right;\">4<\/td>\n<td style=\"text-align: right;\">0<\/td>\n<\/tr>\n<tr>\n<td>Passing generated tests<\/td>\n<td style=\"text-align: right;\">550<\/td>\n<td style=\"text-align: right;\">493<\/td>\n<\/tr>\n<tr>\n<td>Tests that caught injected bugs<\/td>\n<td style=\"text-align: right;\">360<\/td>\n<td style=\"text-align: right;\">316<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The specialized agent completed four more tasks, with no stock-only wins.<\/p>\n<p>The completion rates are much lower than in our internal benchmark. SWE Atlas\nis hard, and there is still a lot to improve.<\/p>\n<h2>What we learned and what comes next<\/h2>\n<p>The clearest gain was reliability. The agent was more likely to deliver unit\ntests that built, passed, and matched the request. The difference was largest\nwhen the prompt was vague or linked to a code change.<\/p>\n<p>Once both setups produced a valid result, neither led every quality measure.\nStock Copilot scored slightly higher on assertions and coverage. The\nspecialized agent scored slightly higher on test hygiene, such as clean\nstructure and avoiding slow or fragile patterns.<\/p>\n<p>These groups contain different tasks. The specialized group includes harder\ntasks that stock Copilot did not complete, so this is not a direct quality\nranking. We see these results as guidance for the next improvements: deeper\nassertions and error-path tests while keeping strong test hygiene.<\/p>\n<p>Efficiency remained close after accounting for completed tasks. The agent used\nabout 3.2% more recorded tokens per completed task. These counts include cached\ninput, so they do not directly represent cost.<\/p>\n<p>These results cover unit tests only. We are exploring where the same workflow\ncould help with other types of testing, but we have no committed plans to\nannounce.<\/p>\n<h2>Try the agent<\/h2>\n<p>The agent, skills, and language guidance are open source in\n<a href=\"https:\/\/github.com\/dotnet\/skills\"><code>dotnet\/skills<\/code><\/a>.<\/p>\n<p>You can use the plugin in GitHub Copilot CLI. It is also available in Visual\nStudio Code and VS Code Insiders through plugin support, which is still in\npreview. We are also working on support for Visual Studio.<\/p>\n<p>Add the marketplace and install the plugin in GitHub Copilot CLI:<\/p>\n<pre><code class=\"language-text\">\/plugin marketplace add dotnet\/skills\r\n\/plugin install dotnet-test@dotnet-agent-skills<\/code><\/pre>\n<p>Restart the CLI. Select <code>code-testing-generator<\/code> from the list of agents. Then\ntry the prompt from the start of this post:<\/p>\n<pre><code class=\"language-text\">Generate unit tests.<\/code><\/pre>\n<p>Review the plan, the tests, and the final checks. You can also ask for tests for\none function, one module, or a specific coverage goal.<\/p>\n<p><div  class=\"d-flex justify-content-center\"><a class=\"cta_button_link btn-primary mb-24\" href=\"https:\/\/github.com\/dotnet\/skills\/tree\/main\/plugins\/dotnet-test\" target=\"_blank\">Explore the dotnet-test plugin<\/a><\/div><\/p>\n<p>Good tests are not only generated. They are planned, built, run, and checked.\nThat is the trust loop we are building.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Meet the open-source polyglot testing agent that learns from a repository, writes unit tests, and checks that they build and pass.<\/p>\n","protected":false},"author":140087,"featured_media":60508,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[685,7781,756,7593],"tags":[8074,46,7869,8195,8194,138],"class_list":["post-60507","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet","category-ai","category-csharp","category-visual-studio-code","tag-agents","tag-c","tag-github-copilot","tag-go","tag-python","tag-unit-testing"],"acf":[],"blog_post_summary":"<p>Meet the open-source polyglot testing agent that learns from a repository, writes unit tests, and checks that they build and pass.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/60507","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/140087"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=60507"}],"version-history":[{"count":1,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/60507\/revisions"}],"predecessor-version":[{"id":60509,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/60507\/revisions\/60509"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/60508"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=60507"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=60507"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=60507"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}