We are excited to announce new capabilities in Azure Test Plans that will enhance your testing workflows. This feature is currently available in Public Preview. You can find instructions at the end of the article on how to request it for your Azure DevOps project.
With this latest release, we are introducing the ability to associate automated tests written in Java/JUnit (Maven and Gradle), JS (Jest) and Python (PyTest) with test cases and then run those tests with the new Azure Test Plan task. This is an addition to the ability to associate tests written in the majority of the .NET supported frameworks, which was until now only supported via Visual Studio Code.
Consequently, this should improve the overall manageability of your software testing processes and the ability to run automated tests directly from the Test Plans, either separately or in combination with manual tests. Here’s how you can take advantage of these new features.
Associate your automated test from a Build Pipeline Run
- First, navigate to Pipelines and choose a pipeline.
- Next, select a build.
- In the “Tests” tab, find the automated test that you want to associate with a test case.
- Select the test and from the command bar, click on “Associate Test Case”.
- Finally, the side panel, find the test case to which the automated test needs to be associated and click on “Associate”.
Run your automated tests with the new Pipeline task
Imagine you have a Test Plan that contains several suites with automated tests for your e-commerce website. One of the test suites contains test cases covering the checkout page of your website. The checkout page might be considered a core part of your site as it is the last step between you and the customer’s cash. Given its importance, you might consider running this suite as part of a pipeline to ensure stable checkout functionality.
Now, you can do this with the newly introduced pipelines task: Azure Test Plan. Add this task to your pipelines and pass a Test Suite as an input variable, which contains all the test cases that have automation associated with them. You can also use a combination of automated and manual tests with this task. The pipeline won’t block until manual tests execute, allowing testers to run them at their convenience. Use the task to run unit and functional tests using the JUnit (Maven and Gradle) and Python (PyTest) frameworks.
Worth noting that, for tests written in C#, you can still leverage the existing, VSTest@3 task.
If you need to execute the tests on an ad hoc basis, outside your regular pipeline runs, you can do this from the test plans. Check out this doc that will walk you through how to run automated tests from test plans.
What’s next?
In the next few months, we will introduce another entry point for associating automated tests to test cases – from within the test case itself. Here is how that experience will look like:
- To start, you can open any test case and navigate to the “Associated Automation” tab.
- Then, under “Associated Automation” click on the Browse button.
- Choose a build in which the desired test was executed and find your test, then click on Associate test.
And there’s more! During Q2 and Q3 of 2025, we will work to onboard even more popular and trending languages and frameworks, such as Python (Robot), Java (Playwright) and more.
Get started now
The feature is now available for public preview, and you can request access to it by submitting this form.
Update (April 30)
In May, we will begin the gradual global release of this feature. Consequently, we are closing the Public Preview registration.
Hi,
When the feature is released for my project, do I need to do sth special to enable it (like prepare a new test plan, new automated tests or rerun the existing ones) or should it be available right away also for the old test runs? Currently I have Azure Basic + Test Plans subscription.
Best regards,
Gosia
Hi,
are there any known issues with this feature? It got enabled a few days ago in one of the ADO organizations that I have access to, so I played around with it a bit. I can successfully run tests based on the test cases defined in the test plan and look at the results in the tests tab of the pipeline run. However, the results are never reported back to the test plan (i.e. the test case outcome is not updated and the test run is not being associated with the test case).
Also: Right now there is a limited set...
For reference, I’ve reported this on Developer Community by now: New AzureTestPlan@0 task does not report results back to test plan. Upvotes appreciated if you are facing the same issue.
Hi there,
There should be no issues. I just run a quick test where I executed a pipeline where I am running PyTests via the Azure Test Plan task. Once the pipeline competed, I can go back and trace the test run on the test case itself. If you keep facing the issues, I suggest you to log a ticket for our support team via your regular channels, or leave feedback on the dev community forum and someone will pick it up for further investigation (https://developercommunity.visualstudio.com/AzureDevOps/report).
I will take the feedback for supporting some generic way of triggering different type of frameworks.
Thanks,Read more
Just wanted to say I have the same issue. Test case does not update in test plans and im also using pytest.
1. When I ‘Associate Test Case’ with my python test, the work item only has ‘Automated test name’ and ‘Automated test storage’ filled in. The ‘Automated test type’ is still blank and not editable.
2. Pipeline is finding the automated test point while it is being run so that is not the issue.
Thanks, someone will pick up the issue raised in the dev community and investigate (https://developercommunity.visualstudio.com/t/post/10920046).
Hi Igor,
Great news 🙂 When the global release will be closed? Is it a matter of days or rather months? I’m looking forward to be able to associate my Python (PyTest) test with AzDO Test Cases but it is still not visible for my subscription/project. Maybe there is a chance to sign up for sooner group for release somehow? Thanks.
Hi Malgorzata,
Thank you for your comment. The timeline we’re looking at is a matter of months rather than days. We’ve experienced a significant influx of requests over the past few weeks, which led us to close the public preview — enabling those projects was quite a resource-intensive work.
I kindly ask for your patience until the GA announcement, after which the feature will be available for your projects as well 🙂 .
Best regards,
Igor
Hi Igor,
This is fanatastic, kudos to you and your team. I would like to point out one of the major use cases I’ve seen for ADO test plans workflows is an E2E suite and a more popular solution for building those E2E is playwright with js/ts framework. Would be great if support can be extended to these.
Hi Fouzan,
The good news is that we will working on providing the support for associating JS Playwright tests in the next couple of months.
What an awesome work! Are other JavaScript frameworks like Vitest also on the roadmap?
In the end it’s just about calling the right commands to get a JUnit XML report, right?
Hi Sebastian,
Thanks for your question. Because ViTest is rather a new framework, first we need to focus on supporting some of the widely used frameworks, that many projects rely on today. In the upcoming months we will be focusing on JS (Playwright), Python (Robot and UnitTest) and a few more.