Supplying Run Time Parameters to Tests

Atin Bansal

Atin

Tests broadly classified as integration tests or functional tests are often in need to connect to external app resource (web application, API front end, or a DB tier) to drive validations. Maintaining lifecycle of such tests from the environment where it’s authored (dev machine with VS) to all the other places where they might be consumed (as part of Build/CI Tests, Build-Deploy-Test workflow, Release workflow) is quite challenging.

As a classic example, consider following piece of functional validation code:

 

 

To effectively reuse such tests throughout the lifecycle it’s important that we provide tooling flexibility for connecting tests to external app resources such as appUrl in the example above. In Visual Studio 2015, the TestRunParameters section in RunSettings file is defined to address this need.

 

 

This value can then be used in tests using TestContext.Properties. Currently, this functionality is only available in MSTest framework. 

 

 

For CI, BDT and Release workflows, we have added the ability to override the value of parameters declared in the runsettings file with those supplied at runtime. These values can be passed to the Visual Studio Test task or Visual Studio Test using Test Agent task using the Override TestRun Parameters field.

 

Figure 1: Specifying value of appUrl in the VSTest task

 

So as to not hardcode the value in the task itself, you can declare a Build variable and then use that at all places in the workflow where you need it.

 

Figure 2: Declaring “Deployment URL” as a Build variable

 

 

Figure 3: Using variable in ‘Visual Studio Test using Test Agent’ task for a BDT workflow

 
Atin Bansal
Atin Bansal

Program Manager, Azure DevOps

Follow Atin   

2 comments

Leave a comment

  • Avatar
    Siddhartha Pandey

    MaxCPUCount can be put in a runsettings file. The runsettings file cant be changed inside your pipeline except for test run params. you need to have this value of maxcpucount in your runsettings. Alternate approach is to have a different runsettings file only for pipeline