Update: October 3,2022
You can now use Azure Load Testing for load testing your applications. Azure Load Testing is a fully managed service to generate high-scale load and identify app performance bottlenecks.When Visual Studio 2019 Preview 1 shipped in early December, we announced our plans to deprecate the load test functionality in Visual Studio. Visual Studio 2019 will be the last version of Visual Studio with web performance and load test features.
To give our customers plenty of notice, I also wanted to let everyone know that we plan on closing down the corresponding Azure DevOps cloud-based load testing service on March 31st, 2020.
Why
Load testing helps you ensure that your apps can scale and do not go down when peak traffic hits. Although we have been shipping our load testing tools and our cloud-based load testing service for many years, unlike our other services their adoption has not been growing. While it is difficult for me to pinpoint one specific reason for this, there are a few contributing factors, including:
- Load testing is typically initiated for seasonal events such as tax filing season, Black Friday, Christmas, summer sales, etc. The classic example I always like to give is the NORAD Santa Tracker
- Load testing requires a certain level of expertise to ensure you have the confidence in the results. This ranges from understanding the application & deployment architecture, designing of load tests, authoring/executing of tests at scale and analyzing the results to identify performance and application bottlenecks.
- We’ve found that very few organizations rely on in-house expertise for this. Instead, most prefer to engage consultants to help them.
Also, frankly, I feel that our offering has fallen behind. When I look around at other offerings in this space (open source as well as commercial offerings that sometimes include consulting services) I honestly feel they are now better suited to meet the needs of our customers.
Please do not think that we are saying that load testing is not important. On the contrary – load testing, done well, remains relevant and will continue to be very important for some of our customers. But I think this need is now best addressed by making sure we have great partner provided or open source load testing options integrated into Azure DevOps.
Timeline for deprecation
-
Visual Studio and Test Controller/Test Agent for on-premises load testing: Visual Studio 2019 will be the last version of Visual Studio with the web performance and load test capability. Visual Studio 2019 is also the last release for Test Controller and Test Agent (installed via the ‘Agents for Visual Studio’ SKU) for setting up a load test rig on-premises on in a virtual machine.
While no new features will be added, load test in VS 2019 will continue to be supported for any issues that may arise during the support lifecycle of the product. As outlined in the product lifecycle and servicing documentation, you can continue to use the on-premises offering and will be fully supported for the next 5 years and an additional 5 years of extended support is also available should you need it. Different Visual Studio versions can be installed side-by-side on the same machine. This means that you can continue to use Visual Studio 2019 to maintain your on-premises load tests, while using a newer Visual Studio version when it becomes available in the future for other development needs.
Visual Studio web performance tests (.webtest files) are tied to the load test functionality and is also deprecated. While it was never designed for it specifically, I know of a few customers who have used .webtest’s for other purposes such as for running API tests. I would encourage them to take a look at some of the other API testing alternatives that are available in the market that would better serve their needs. SOAP UI is a free, open source alternative to consider and is also available as a commercial option with additional capabilities.
-
Shutting down the cloud-based load testing service: Our cloud-based load testing service will continue to run through March 31st, 2020. You can continue to use all the experiences powered by this service without interruption until then. After the service goes offline, you can continue to use the tests as outlined below while you evaluate alternatives.
- Running a load test in the cloud using Visual Studio (both auto-provisioned agents and self-provisioned ‘bring-your-own’ agents): You can continue to run the tests but self-hosted with Test Controller/Test Agent option outlined above.
- Using the web-based experience for load testing in Azure DevOps (URL-based, HTTP Archive, and Visual Studio webtest): You can export these tests to Visual Studio load test and continue to run them self-hosted with the Test Controller/Test Agent option.
- Using Apache JMeter tests in Azure DevOps : You can continue to run the tests with a self-hosted JMeter setup of your own or using any 3rd party commercial services that support it.
The cloud-based load testing service is also used in the following places, which will no longer be available after March 31st, 2020.
- Running load tests in Azure Pipelines using the load testing tasks (Cloud-based load test task, Apache JMeter task and the cloud-based web performance test task: These tasks will stop working after March 31st, 2020 and will be removed.
- Cloud-based load tests in CI/CD pipelines will run until March 31st, 2020. After that you need to self-host them and run using the Test Controller/Test Agent setup invoking the tests through the mstest.exe command-line.
- Running performance tests using the Azure portal (performance test option in App Services and Application Insights blades): These options will no longer be available after March 31st, 2020.
Load testing Alternatives
There are many alternatives, both free and commercial tools that you can consider. For instance, Apache JMeter is a free, popular open source tool with a strong community backing. It supports many different protocols and has rich extensibility that can be leveraged to customize the tool to your needs. Many commercial services such as Blazemeter support running Apache JMeter tests.
If you use code based tests for load testing and .NET is your platform of choice then tools such as Neoload, Micro Focus Silk Performer and Micro Focus Load Runner are options to consider.
In addition, extensions from several load test vendors such as SOASTA (now Akamai CloudTest), Apica Loadtest and Load Impact are available in the Azure DevOps and Azure marketplace.
More Information
Deprecating features is always hard and therefore not something we ever do lightly. We recognize that this announcement will be disruptive for the people that are using the service today and we will be working with affected customers to ensure they are aware of the changes. Premier support for Enterprise can be engaged for help with migrating tests to alternatives – they can be reached via email on premdevinfo@microsoft.com
If you have any further questions or concerns then please send us an email at vsoloadtest@microsoft.com or contact me directly.
I am a little confused after reading this part in the memo.
"Running a load test in the cloud using Visual Studio (both auto-provisioned agents and self-provisioned ‘bring-your-own’ agents): You can continue to run the tests but self-hosted with Test Controller/Test Agent option outlined above."
What does this mean? If I have load test agents, controllers created in Azure - will I be able to continue to use it if I do not use Visual Studio...
Hi Jamie,
On Visual Studio 2019 Performance Testing tool, have below few questions.
1) In Visual Studio 2019, How Long do we have support for On Premise load test feature using Test Controller & Agents?
2) .Webtest support will be for another 5 years along with On-premises?
Thanks,
Mahendra
Sad, our team just discovered this tool in DevOps and we were pumped to integrate this into our release process. Wish Microsoft would give it a little more time to be adopted by teams. I think the biggest issue is visibilty, there are some many features to DevOps we didn’t notice this feature till recently.
In azure Portal I’ve implemented extensive Synthetic monitoring via the Availability test functionality within app’ insights. Webtests are fundamental in allowing complex journeys to be captured.
So I ask, will these availability tests also be deprecated on in March 2020?
Kind Regards,
Timur
Hi Timur,
No, this announcement doesn’t cover Availability tests. When/if we decide to deprecate Multi-step Availability tests (note, Single URL will not be affected) there will be an additional announcement. There will be another year after announcement while Multi-step Availability tests are supported.
Thank you!
Zaki
It's a shame but not surprised as it hasn't had any serious development since 2010 :| no body has written any serious addons for it for over 5 years. Is it any surprise it fell behind I'm sure most of the rest of MS's test solutions will follow suit fairly quickly as most of them have had no investment for about as long :(
But then anyone who ever expected MS to take QA seriously,...
When I started to work with AzureDevOps I literally fell in love with al the services, I think that the hardest thing of leaving the load test service is that I use AzureDevOps to share results for unit tests and load tests, metrics, and graphs, now I will not be able to have these results In one tool..
Is there any chanse to keep displaying load test results in this way? even If the run...
Can I get some clarity on the wording in this notice?I only use the web-based experience for load testing. Above it seem like I can keep using it past 31st of March 2020?
“After the service goes offline, you can continue to use the tests as outlined below while you evaluate alternatives.” web-based experience … is listed under.So can I use it going forward after the deadline or not?
We are exploring an idea for keeping our Visual Studio-based performance test framework alive which might provide a pathway for others seeking to retain the use of their VS test assets. I wrote a brief outline of our idea over on this thread with the request for Microsoft to open source: https://developercommunity.visualstudio.com/idea/490150/open-source-performance-testing-tools-and-allow-co.html.
Please get in touch if you are interested in exploring that idea further.
Should probably put an end of life notice on https://docs.microsoft.com/en-us/learn/modules/load-test-web-app-azure-devops/.
It´s a pitty, I was about to make a course about this tool but a think i´ll give up.