Supplying Run Time Parameters to Tests

Atin Bansal

Tests broadly classified as integration tests or functional tests are often in need to connect to external app resource (web application, API front end, or a DB tier) to drive validations. Maintaining lifecycle of such tests from the environment where it’s authored (dev machine with VS) to all the other places where they might be consumed (as part of Build/CI Tests, Build-Deploy-Test workflow, Release workflow) is quite challenging.

As a classic example, consider following piece of functional validation code:



To effectively reuse such tests throughout the lifecycle it’s important that we provide tooling flexibility for connecting tests to external app resources such as appUrl in the example above. In Visual Studio 2015, the ***TestRunParameters ***section in RunSettings file is defined to address this need.



This value can then be used in tests using TestContext.Properties. Currently, this functionality is only available in MSTest framework. 



For CI, BDT and Release workflows, we have added the ability to override the value of parameters declared in the runsettings file with those supplied at runtime. These values can be passed to the Visual Studio Test task or Visual Studio Test using Test Agent task using the Override TestRun Parameters field.


Figure 1: Specifying value of appUrl in the VSTest task


So as to not hardcode the value in the task itself, you can declare a Build variable and then use that at all places in the workflow where you need it.


Figure 2: Declaring “Deployment URL” as a Build variable



Figure 3: Using variable in ‘Visual Studio Test using Test Agent’ task for a BDT workflow



Discussion is closed. Login to edit/delete existing comments.

  • Aryamol Jacob 0

    I want to override the maxCpuCount in run settings file to 1 to run tests sequentially.
    How can I do it in azure pipeline yaml file, for a particular job

  • Siddhartha PandeyMicrosoft employee 0

    MaxCPUCount can be put in a runsettings file. The runsettings file cant be changed inside your pipeline except for test run params. you need to have this value of maxcpucount in your runsettings. Alternate approach is to have a different runsettings file only for pipeline

Feedback usabilla icon