After publishing **Load Testing from different geographies with Cloud-based load testing **I have have had a couple of people ask how do you schedule Load Tests.
Depending on what type of load testing you are doing there are two ways to do this.
- For on-premises Load Test you will need to use the Command line (see directions for doing this at the bottom of this post)
- For Cloud Load Testing we make this simple with the new Build System
While you could use the command line for both on-premises and Cloud-based Load Test (see http://blogs.msdn.com/b/visualstudioalm/archive/2015/08/24/cloud-load-test-support-in-mstest-exe-command-line-and-xaml-builds.aspx) the new Build System for Cloud-based Load Test makes this much easier.
Scheduling a Cloud-based Load Test as part of the a Build
In addition to being much easier, the new build system in Visual Studio Online as a scheduling engine also has the added benefit of being able to connect your load test into an entire workflow and able to kick off release pipelines with approvals and end to end reporting. To get started:
1. Create a new Build
Navigate to the Build hub and create a new build
In the Build templates select empty…as we will only need the Load Test Tasks
2. Create a Service End Point
Cloud-based Load Testing execute from a remote agent and must have access and “know” which VSO Account to publish the results back to. This is done through Service Endpoints. To create a Service Endpoint for Load Test you must go to control panel add a Generic service set to the VSO Account you want the load test to run from.
**REMEMBER THIS REQUIRES A PAT OR ALTERNATE CREDENTIALS. **
I always seem to forget this! Directions for setting alternate credentials and creating a PAT can be found here:
http://roadtoalm.com/2015/07/22/using-personal-access-tokens-to-access-visual-studio-online/
3. Setup the Load Test Tasks
Add both the Quick Load Test Task and Load Test Task
For the Quick Web Performance Test simply up the parameters for your quick performance test:
For the regular Load Test Task you will have to create a Load Test Using Visual Studio Enterprise and Check it in to the repo we are building
Note the Testing settings file is typically in the root of the SLN.
3. Scheduling the Build
Under the triggers tab of the build definition select Triggers and select the “Scheduled” option.
Scheduling an on-premises Load Test
Sean Lumley’s post below will show you how to run a web test or load test from the command-line instead of from within Visual Studio.
That post and the use of the Windows Task Scheduler
Most of the time you will want to run you web or load test from within visual studio, but you can run them from a command-line. Maybe you have a set of 5 web tests that you want to run once a day. You could configure a scheduled task to run a batch file which contained ran the 5 web tests. The command line executable that is used to run tests is called mstest.exe and is installed when you install either VS Team Suite or VS Team Test SKU. Here is a MSDN help link for mstest: http://msdn.microsoft.com/en-us/library/ms182486.aspx
The easiest way to get started with mstest is to start a visual studio command prompt. Go to Start Menu -> All programs -> Microsoft Visual Studio v9.0 -> Visual Studio Tools -> Visual Studio 9.0 Command Prompt. This will launch a command window which has mstest.exe in its path. So type mstest /? This will show each of the available options. Let’s run through a few options of how to run tests.
1) Running a single WebTest
First let’s run a simple web test. To do this, we just need to specify the /TestContainer argument. A .webtest file or .loadtest file is considered a test container in the same way that a dll which contains unit tests is a test container. So if we have a web test called WebTest1.webtest, you would use the following command to run the web test:
mstest /TestContainer:WebTest1.webtest
This will produce output similar to the following:
Loading WebTest1.webtest…
Starting execution…
**Results Top Level Tests
Passed webtest1.webtest**
1/1 test(s) Passed
**Summary
Test Run Completed.**
**Passed 1
Total 1**
Results file: C:Documents and SettingsslumleyMy DocumentsVisual Studio 10ProjectsTestProject3TestProject3TestResultsslumley_SLUMLEYTCM 2008-12-22 13_38_14.trx
2) ****Running Multiple WebTests
Now let’s execute 2 web tests from one call to mstest. You can specify multiple test container arguments on one command-line. So say we wanted to execute WebTest1.webtest and WebTest2.webtest. You would use the following command-line:
mstest /TestContainer:WebTest1.webtest /TestContainer:WebTest2.webtest
This will produce output similar to the following:
Loading WebTest1.webtest…
Loading WebTest2.webtest…
Starting execution…
Results Top Level Tests
-—— —————
Passed webtest1.webtest
Passed webtest2.webtest
2/2 test(s) Passed
Summary
-——
Test Run Completed.
Passed 2
-——–
Total 2
Results file: C:Documents and SettingsslumleyMy DocumentsVisual Studio 10P
rojectsTestProject3TestProject3TestResultsslumley_SLUMLEYTCM 2008-12-22 13_4
3_04.trx
3) ****Specifying deployment items
One of the big differences with running from the command line is that you are not within a visual studio project. When you run a web test from within Visual Studio, we will try to determine what needs to be deployed with the web test in order for it to run. For example, maybe you have written a custom validation rule or a extraction rule. When you run the test from within visual studio, we will look at the references of the test project and try to figure out what needs to be deployed.
When you are running from the command-line, you will need to be very explicit about what you deploy. So if you have a dll that needs to be deployed for a test to run, you will need to inform mstest. The way you do this is by specifying /runconfig parameter. One of the sections of a run config is deployment items. Use visual studio to create a run config. Then to run a test with a runconfig, you would have a command-line similar to the following:
Mstest /TestContainer:WebTest1.webtest /RunConfig:NewRunConfig.testrunconfig
4) ****Executing a test on a controller/agent setup
When in VS, to run a web or load test on a controller/agent setup you need to use a test run configuration. You would open the run config, specify to run the test remotely and specify a controller name. It works the same way from the command-line. You will need to create a run config that has the correct settings. Then you run it the same way as you do when specifying deployment items:
Mstest /TestContainer:WebTest1.webtest /RunConfig:NewRunConfig.testrunconfig
5) ****Running a coded web test
Running a coded web test is similar to running a unit test. You will set the /testcontainer argument to the dll which contains the coded test.
mstest /TestContainer:TestProject1.dll
One thing you will notice when running a test this way, is that mstest will execute all of the tests within the dll.
6) ****Executing a specific coded test in a dll
So if you want to execute just one test within a dll, you would need to use the /test argument. For example, to run WebTest1Coded, you would use the following command line:
mstest /TestContainer:TestProject1.dll /Test:WebTest1Coded
If you wanted to execute 2 tests, you can specify multiple /Test arguments. For example:
mstest /TestContainer:TestProject1.dll /Test:WebTest1Coded /Test:WebTest2Coded
7) ****Executing a load test
This is the same as executing a web test except you need to specify the .loadtest file
mstest /TestContainer:LoadTest1.loadtest
When the test completes, you can either open the trx file to get to the load test results or you can use the manage load test results dialog form within VS. Here is a blog on that: http://blogs.msdn.com/slumley/pages/managing-load-test-results.aspx
8) ****Specifying the results file name
You will notice that the results file(trx file) gets a unique name which contains user, machine and a timestamp. If you want to specify the name of the results file and where it is generated, you can use the /resultsfile parameter. For example, to create a results file called MyResults.trx and have it saved to c:results, I would do the following:
mstest /TestContainer:WebTest1.webtest /resultsfile:c:resultsMyResults.trx
Hopefully this post shows you how to make use of the mstest tool.
0 comments