Easily provision the required resources in Microsoft Azure using the Azure Resource Manager templates, and deploy Web applications and SQL Server Databases to the Azure Virtual Machines (VMs), and run tests that are automatically distributed across VMs using the Visual Studio Test Agent. Once the application’s functionality is verified, deploy the application to on-premises machines using the same IIS/SQL tasks that were used to deploy the app to the Azure VMs. The tasks provide an easy way to override the database connection strings, and any of the parameters that are changing between development, testing, staging and production.

The blog explains how to build the app and deploy it to the Azure VMs using Build and Release Definitions in Visual Studio Team Services (VSTS) and Team Foundation Server (TFS), and then to deploy the app to production on-premises.

Service Endpoints

To deploy to Azure, an Azure subscription has to be linked to the VSTS/TFS account using the Services tab in the Account Administration section. The sample app is in the FabrikamFiber GitHub repository. Using the Services tab we will link the GitHub repo to the VSTS/TFS account.

Add Azure Resource Manager Subscription

Add the Azure subscription to use in the Build tasks by opening the Account Administration screen (gear icon on the top-right of the screen) and then click on the Services tab. Select** Azure Resource Manager** from the Add New Service Endpoint drop-down. Create a Service Principal as per the steps provided here. Also, note the GitHub in the **New Service Endpoint **drop-down, as this will be used to setup a GitHub repository connection in the section below.

 Figure: Opening the Account Administration Screen.

Figure: Selecting an Azure connection from the Add New Service Connection.

Fill in the required details in the Add Azure Resource Manager Subscription dialog. Note that the Create New SPN functionality is coming soon and will ease the process of creating Service Principals.

Figure: Adding the details of the Service Principal in the Add Azure Resource Manager Subscription dialog box.

Add New GitHub Service Connection

Select** GitHub from the Add **New Service Endpoint drop-down, and fill in the parameters as shown below. Note that the Connection Name can be anything that you like but the token has to be as listed below. After entering the parameters click on OK.

Choose Authorization: Personal access token

Token: 13b56d00b0b78d1c26ab359df7a44014085efa37

Connection Name: FabrikamFiber GitHub

Figure: Adding the GitHub repo details in the Add New GitHub Service Connection dialog box.

Automation Agent

An automation agent, also called as build agent, has to be deployed on a machine to build the code and to run the tasks. The agent is xcopy deployable, and does not need any installation. The agent can be deployed on machines behind the Firewall and also on Azure VMs. The agent uses the HTTPS protocol to communicate with VSTS/TFS and can work across proxies and firewalls. VSTS also provides Hosted automation agents that can be used to build code and to deploy the application. The Hosted agents are made available on-demand and run on Azure.

Deploy the Windows build agent as per the instructions in the Build docs. The following are some of the issues that one should keep in mind while deploying the build agent:

1. As described in the Build docs, either use the default queue or create a new queue.
2. Provide the rights to the account, under which the Build agent is running, in the VSTS’s or TFS’s Control Panel named Agent Pools. This account should be added to the Agent Pool Service Accounts group.
3. Download and configure the agent by following the instructions.
4. The tasks described in the blog needs Visual Studio 2015 and Azure PowerShell to be installed on the build agent. Visual Studio 2015 is needed for building the code and Azure PowerShell is needed to deploy Azure Resource Groups. Azure PowerShell can be installed from Azure PowerShell Installer v1.3.0.

IIS Web App Deployment  Using WinRM Extension

The extension is needed because it has the tasks to deploy IIS Wweb apps and to deploy SQL Server database. Install the extension from the VSTS marketplace. For TFS, download the extension and install it.

Dev/Test Applications on Azure

Follow the directions below to build the app in  the Build hub and to deploy it to Azure VMs from the Release hub.

Create Build Definition & Link the GitHub Repository

Open the BUILD hub and choose the icon to create a new build definition.

Figure: Creating a new Build Definition.

In the Build tab of the Create new build definition dialog, select an Empty definition and choose Next.

Figure: Selecting the Build Definition template.

Select GitHub as the Repository Source, and then click on the Create button.

Figure: Selecting the Build Definition template.

In the Build Definition, click on the Repository tab and select the fabrikamfiber/customerservice Repository.

Figure: Setting the repository.

In the Definition, click on the Build tab, and then on  Add Build Step button to add tasks to the Definition.

In the Add Tasks dialog, click on the hyperlink All, and then add the tasks – NuGet Installer, Visual Studio Build & Copy and Publish Build Artifacts. The tasks are sorted alphabetically, and the Add button in front of the task, has to be clicked to add the task to the definition.

Figure: Add the tasks to the Build Definition.

Leave the NuGet Installer task with it’s default values. Select the Visual Studio Build task in the Definition, and in the MSBuild Arguments parameter, enter the value as shown below in a single line. This will build the zip file that can be deployed using Web Deploy later on in a release definition. Leave everything else at their default values:

/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true  /p:PackageLocation=”$(build.sourcesDirectory)pkg”

 

Figure: Defining the parameters for the Visual Studio Build task.

Select the Copy and Publish Build Artifacts task in the Definition and enter or select the following values for the parameters. The built app code has to be published as an artifact to use in the release definitions:

Copy Root: $(build.sourcesDirectory)

Contents: Note that these are multi-line values.

**pkg **FabrikamDBbinRelease **FabrikamFiber.Web.TestsbinRelease **DeployTemplate

Artifact Name: FabrikamCustomerService

Artifact Type: Server

Figure: Defining the parameters for the Copy and Publish Build Artifacts task.

Click on the Save button in the Definition, and in the Save dialog, enter the name of the definition as Fabrikam.CustomerService.CI, and then click the OK button.

After saving the definition, click on the Queue Build button in the Definition, and then click OK in the Queue Build dilaog, to build the app, and to publish the artifacts

.

Create Release Definition to Deploy and Test the App

The Azure Resource Group Deployment task is used to deploy Resource Groups in Azure using the Azure Resource Manager templates.

Figure: Azure Resource Group Deployment task in the Build Definition to deploy a Resource Group on Azure.

The different parameters of the task are as described below:

Azure Subscription: Select the Azure Subscription where the Resource Group will be deployed. This is a dropdown of the subscriptions that have been added in the Services tab.

Resource Group, Location: The name of the Resource Group and its location. If this is an existing Resource Group then the task will update the Resource Group with the Resources specified in the Azure template. If no Resource Group with the name exists in the Subscription then a new one will be created.

Template and its Parameters: The templates and the templates parameters file are the Azure templates available at GitHub and in the Azure gallery. To get started immediately, use this template that is available on GitHub. These files can be either checked in the Version Control of they can be part of the build itself. If the files are part of the Build, use the pre-defined system variables provided by the Build to specify their location. The variables to use are $(Build.Repository.LocalPath), if the templates are checked-in but are not built, or $(Agent.BuildDirectory), if the templates are built as part of the solution. Be sure to specify the full path like $(Build.Repository.LocalPath)Azure TemplatesAzureRGDeploy.json. Wildcards like ***.json or ***.param.json are also supported and there needs to be only file that matches the search pattern at the location. If more than one file matches the search pattern then the task will error out.

Override Template Parameters: The Override template parameters is used to override the parameters, like –storageAccountName azurerg –adminUsername $(vmusername) –azureKeyVaultName $(fabrikamFibre).

Advanced Deployment Options: This section can be confusing if one does not understand the rationale behind the options. The following information will help in understanding the parameters and the proper use of them:

• Refresh SAS Token: Azure provides in-built ability to deploy applications using Resource Extensions (RE) for PowerShell-DSC or Custom Script Extension for any of the scripting languages for Linux (Python, Shell scripts). The scripts and application files need to be available on an Azure storage account because the RE downloads them to the VMs and then runs them. The storage accounts are normally secured, and a Shared Access Signature (SAS) token is needed to access them. More information about the Azure SAS token is here. This SAS token has an expiration date and the storage account cannot be accessed after the SAS token expires. This will break the Continuous Integration (Build) pipeline because the task will fail as it cannot access the storage account. The Refresh SAS token parameter refreshes the SAS token in the Azure template prior to sending it to Azure each time the Build is run. To do that it needs the name of the parameters for Azure Blob and SAS token as they have been specified in the Azure template parameters file. Multiple of these can be provided in Azure Blob Parameter and SAS Token Parameter by using the semi-colon delimiter like azurestore1; azurestore2; azurestore3. For example, the template parameters file has the following key-value pairs for Azure Storage account and SAS token. Then fill in azureStorage in the Azure Blob Parameter in the Azure Resource Deployment task and sasToken in the SAS Token Parameter.Note that Refresh SAS token is optional and only needed if deployment is being done to the Azure VMs using the Azure Resource Extension. As demonstrated in this blog, if the PowerShell on Target Machines task is being used to deploy the app then this Refresh SAS token is not needed.

"azureStorage": { "value": "fabrikamfibre" }, "sasToken": { "value": " ?sv=2014-02-14&sr=c&sig=Dj1QOJups1%2Bf%2Beq989j%2FomowbmNS8Q2Qmf5eJEU9FGg%3D&st=2015-03-26T18%3A30%3A00Z&se=2015-04-03T18%3A30%3A00Z&sp=r" }

• Virtual Machine Credentials: These are the Admin credentials for the Azure VMs that have been specified in the Azure template. The VM credentials are stored securely in the Deployment service, so that any subsequent tasks that need to access the VMs will not have to take this as an input. In this walkthrough, the tasks that need the credentials are Azure File Copy, PowerShell on Remote Machines, and Visual Studio Test Agent Deployment.
• Test Certificate: The test certificate parameter requires background knowledge that is explained below. This parameter will be explained post the sections below.

Setting-up WinRM HTTPS on Azure VMs: Tasks like Azure File Copy, PowerShell on Target Machines, Visual Studio Test Agent Deployment run on the Build Agent machine and copy files or deploy apps to Azure VMs using the WinRM HTTPS protocol. For these tasks to work properly the WinRM HTTPS port (default port is 5986) needs to be opened and configured properly on the VMs. Opening the ports and configuring them with the certificates is done using the Azure templates. The sample template uploaded on GitHub shows how to enable the WinRM HTTPS port on Azure VMs and map them to a Public IP using the Azure resource provider’s wiz. Network Interfaces, Load Balancers and Virtual Machines. In addition, it also shows how to specify the Azure Key Vault and its secret, to download and install the certificate on the VM.

Azure Key Vault and Test Certificates: Azure Key Vault provides a secure way of storing certificates in Azure that can be easily downloaded and installed on the Azure VMs to enable WinRM HTTPS communication protocol. To create test certificates and to upload the certificates to Azure Key Vault follow the steps given below. Note that the Azure Key Vault can be in its own resource group and does not need to be in the resource group that is being dynamically created using the Azure Resource Group Deployment task. This way, once a certificate has been uploaded to the Azure Key Vault it can be reused across different Azure resource group deployments.

• Run the following commands from the developer command prompt to create the test certificate. Replace the Common Name (CN) with the Resource Group name and the password with the real password.

makecert -sv armtest.pvk -n "cn=*.westus.cloudapp.azure.com" armtest.cer -b 06/06/2015 -e 06/06/2016 -r
pvk2pfx -pvk armtest.pvk -spc armtest.cer -pfx armtest.pfx -po password

• For uploading the certificate to the Azure Key Vault follow the steps given below:
  • Download the KeyVaultUrl.ps1 available with this blog to the local disk. Remember to unblock the zip file after downloading it, else the KeyVaultUrl.ps1 will not run.
  • Open Windows Azure PowerShell
  • Change directory to where the KeyVaultUrl.ps1 file was copied to.
  • Run the following commands:

 Add-AzureAccount 
 Select-AzureSubscription -SubscriptionName <name> 
 Switch-AzureMode AzureResourceManager 
 Import-Module .KeyVaultUrl.ps1 –Force

• To generate the secret Id run the command given below:

$a = Generate-KeyVaultUrl -resourceGroupName <resourceGroupName> -keyVaultName <KeyVaultName> -secretName <secretName> -location <location> -certificatePath <certPath> -password <password>

• Example:

$a = Generate-KeyVaultUrl -resourceGroupName nikhilnew13 -keyVaultName nikhilkeyvalut13 -secretName test -location "West US" -certificatePath "D:certArmTest.pfx" -password password

• Secret Id will get printed as well as get stored in $a variable and it will be similar to – https://fabrikamvalut.vault.azure.net:443/secrets/fabrikam/256892c857714ec0bd76529147cf6b11

Test Certificate: The Build Agent uses the WinRM_HTTPS communication protocol to deploy apps to the Azure VMs. For secure communication, certificates are used and they need to be installed on the VMs. For developing and testing apps, usually test certificates are used, and by checking-off the Test Certificate parameter, the trusted certificate authority (CA) validation is skipped. Note that the Common Name (CN) of the certificate has to be same as that of the Fully Qualified Domain Name (FQDN) of the VM. If the CN and FQDN different then the task will error out.

Azure File Copy

The task is used to copy application files and other artifacts that are required to install the application on Azure VMs like PowerShell scripts, PowerShell-DSC modules etc.

Figure: Azure File Copy for copying files to Azure blobs or Azure VMs.

The task provides the ability to copy files to an Azure blob or directly to Azure VMs. Even when the target is Azure VMs, Azure blobs are used as an intermediary and the files are copied to it first and then downloaded to the VMs. The tasks uses AzCopy, the command-line utility built for fast copying of data from and into Azure storage accounts. The different parameters for copying files to Azure VMs are as described below:

Azure Subscription: The name of Azure subscription, where the Azure storage account is located. The storage account is accessed using the stored credentials of the Azure account in the Services tab.

**Source: **The source of the files. As described above using pre-defined system variables like $(Build.Repository.LocalPath) make it easy to specify the location of the build on the Build Automation Agent machine. The variables resolve to the working folder on the agent machine, when the task is run on it. Wild cards like ***.zip are supported and should resolve to a single file or a folder.

Storage Account: The name of an existing storage account in the Azure Subscription specified earlier.

**Destination: **The target for copying the files and is either an Azure blob or VMs. The section below details the parameters that need to be filled-out if the target is Azure VMs. Note that for copying the files to VMs, they are first copied to an automatically generated container in the Azure storage account, and then from there to the VMs. The container is deleted after the files are copied successfully to the VMs.

• Resource Group: Name of the resource group that contains the Azure VMs.
• Select Machines By: **The parameter is used to copy the files to a subset of VMs and the subset can be specified by the names of the machines or the tags on them. ** 
• Copy to Machines: If copying to a subset of machines provide a comma separated list of the VMs here, else if using tags then provide the tags in the format Role:Web; OS:win7. The default is to copy to all the VMs in the Resource Group.
• Destination Folder: The folder in the Azure VMs where the files will be copied to. Environment variables are also supported like $env:windir, $env:systemroot etc. An example of the destination folder is $env:windirFabrikamFibreWeb.
• Clean Target: Checking this option will clean the destination folder prior to copying the files to it.
• Copy Files in Parallel: Checking this option will copy files to all the VMs in the Resource Group in-parallel, hence speeding up the process of copying.

**Destination: **If the target is Azure blob then the following parameters need to be filled out.

• Container Name: The name of the container where the files will be copied to. If the container does not exist then a new one will be created with the name provided in this parameter.
• Blob Prefix: A prefix for the Blobs that can be used to filter the blobs like appending the Build number to the blobs, so that all the blobs with the same build number can be downloaded from the Container.

PowerShell on Target Machines

The task is used run PowerShell on the target machines. The task can run both PowerShell scripts and PowerShell-DSC scripts. For PowerShell scripts, PowerShell 2.0 is needed on the machines and for PowerShell-DSC scripts Windows Management Framework 4.0 needs to be installed on the machines. WMF 4.0 ships in-the-box in Windows 8.1 and Windows Server 20012 R2.

Figure: PowerShell on Target Machines task runs on the machines to install applications

The different parameters of the task are explained below:

• Machine Group: The name of the Azure Resource Group.
• Select Machines By: **The parameter is used to specify the subset of VMs, where the PowerShell will be run and the subset can be specified by the names of the machines or the tags on them. ** 
• Deploy to Machines: If deploying to a subset of machines provide a comma separated list of the VMs here, else if using tags then provide the tags in the format Role:Web; OS:win7. The default is to deploy to all the VMs in the Resource Group.
• **PowerShell Script: **The location of the PowerShell script on the target machine like c:FabrikamFibreWeb. Environment variables can be also used like %systemdrive%Web or %temp% FabrikamFibreWeb etc.
• **Script Arguments: **The arguments needed by the script, if any provided in the following format -applicationPath $(applicationPath) -username $(vmusername) -password $(vmpassword).
• Initialization Script: The location of the data script that is used by PowerShell-DSC and the location has to be on the target machine. It is advisable to sue arguments in place of the initialization script.
• Advanced Options: The advanced options provide more fine-grained control on the deployment.
• Run PowerShell in Parallel: Checking this option will execute the PowerShell in-parallel on all VMs in the Resource Group.
• Session Variables: Used for setting-up the session variables for the PowerShell scripts and the input is a comma separated list like $varx=valuex, $vary=valuey. This is mostly used for backward compatibility with the earlier versions of Release Management product and it is advisable to use arguments in place of the session variables.

Visual Studio Test Agent Deployment

The task is used to deploy the Visual Studio Test Agent on the machines. The task runs on the Build Agent machine and uses PowerShell to deploy the Test Agent to the machines.

Figure: Visual Studio Test Agent Deployment task runs on the machines to install and configure the test agent.

The different parameters of the task are explained below:

• Machine Group: The name of the Azure Resource Group.
• Select Machines By: **The parameter is used to specify the subset of VMs, where the test agent will be installed, and the subset can be specified by the names of the machines or the tags on them. ** 
• Deploy to Machines: If deploying to a subset of machines provide a comma separated list of the VMs here, else if using tags then provide the tags in the format Role:Web; OS:win7. The default is to deploy to all the VMs in the Resource Group.
• **Username: **The credentials that the test agent will use to run on the machine.
• **Password: **The password of the user.
• **Interactive Process: **Checking this option will configure the test agent to run as an interactive process and this is needed for running the Coded UI tests.
• **Test Agent Location: **The test agent is downloaded and installed on the machines from http://go.microsoft.com/fwlink/?LinkId=536423. The default behavior can be overridden by manually downloading the test agent and then providing the test agent’s location as a UNC or local path.
• **Update Test Agent: **Checking this option will update the test agent and its configuration.
• Enable Data Collection Only: Checking this option will configure the test agent to only collect data, and no tests can be run using the test agent. Useful when data needs to be collected from the application-under-test.

Visual Studio Test using Test Agent

The task is used to run tests on the machines where the test agent has been deployed.

Figure: Visual Studio Test using Test task runs tests on the machines where the test agent has been deployed.

The different parameters of the task are explained below:

• Test Machine Group: The name of the Azure Resource Group where the tests will be run. The test agent should have been already deployed and configured on the VMs using the Visual Studio Test Agent Deployment task.
• **Test Drop Location: **The folder on the VMs where the test binaries are located. Environment variables can be also used like %systemdrive%Tests or %temp%Tests etc.
• **Test Assembly: **The test binaries for running the tests. Wild cards can be used like ***FabrikamTests*.dll that will use all test assemblies with FabrikamTests in their name.
• **Test Filter Criteria: **Used to specify the test filter criteria like ‘Owner=Tom&Priority=0’.
• **Platform: **The platform against which the tests will be reported like any cpu or x86 or x64.
• **Configuration: **The configuration against which the tests will be reported like any debug or release.
• Run Settings File: The location of the run setting file on the Build Automation Agent machine. Predefined variables like $(Build.Repository.LocalPath) can be also used.
• Override Test Run Parameters: Used to override parameters in the TestRunParameters section of the run setting file like WebAppURL=$(appURL);Port=8080. Here the $(appURL) is a variable that has been defined in the Variables tab of the Definition.
• **Test Configurations: **Used to associate a test case filter against a test configuration ID. The syantax is :; DefaultTestConfiguration:, like FullyQualifiedName~Chrome:12.
• **Code Coverage Enabled: **Checking this option will enable code coverage during the testing.
• **Application Under Test Machine Group: **The name of resource group where the Application under Test is running. If the same resource group is being used for deploying applications and running tests then the name of Test Machine Group and Application Under Test Machine Group will be same.

Machine Group Actions

The task is used to start/stop/restart/delete Azure Resource Groups.

Figure: Machine Group Actions task to start/stop/restart/delete Azure Resource Groups.

The different parameters of the task are explained below:

• Azure Subscription: The Azure Subscription, where the resource group is located.
• **Machine Group: **The name of the Azure resource group.
• **Action: **The action to be performed on the resource group wiz. start, stop, restart, or delete. The block and unblock actions currently do not work with Azure resource groups and once that has been fixed, the updates will be posted to the blog.
• Select Machines By: **The parameter is used to specify the subset of VMs on which the action will be performed, and the subset can be specified by the names of the machines or the tags on them. ** 
• Apply to Machines: If deploying to a subset of machines provide a comma separated list of the VMs here, else if using tags then provide the tags in the format Role:Web; OS:win7. The default is to deploy to all the VMs in the Resource Group.

KeyVaultUrlScript.zip