November 16th, 2017

Release Gates: Releases with continuous monitoring built in

Shashank Bansal
Principal Product Manager

Continuous monitoring is an integral part of safe deployments and DevOps pipelines. Ensuring the application in a release is healthy after deployment is as critical as the success of the deployment process. Enterprises adopt various tools for automatic detection of application health in production and for keeping track of customer reported incidents. Manual approvals are frequently used to gather and incorporate feedback for deployments to an environment. The user who approves the deployment manually monitors all the health signals before promoting the release. For instance, we commonly encounter the following conditions:

  • Ensure there are no active blocking bugs before starting the release
  • Ensure that monitoring system reports a healthy application before starting the next stage of the release
  • Ensure all incidents reported from the customers are resolved before starting the next stage of the release

Release Management in Visual Studio Team Services (VSTS) now supports integrated continuous monitoring with release pipelines. You can now use this capability to automatically detect if the release is ready to be promoted to the next environment. The system repeatedly queries  the health signals for the application until all of them are successful at the same time before continuing the release. With this, you can now act on feedback early in the release process and stop damage downstream.

Preview

This feature is currently in preview. To enable it for the account, you can go to Preview features under your profile menu and then toggle the Approval gates in releases feature on.

Enable gates preview feature

Gates

A release definition specifies the end-to-end release process for an application to be deployed across environments. A deployment process is defined for each of the environments, and the deployments into or out of an environment are currently controlled using approvals. Gates define the monitoring services to query and evaluate for deployments into or out of an environment. Each gate can monitor one or more health signals corresponding to a monitoring system of the application. Built-in gates are available for “Azure monitor (application insight) alerts” and “Work items”. You can also integrate with other systems using the flexibility offered through Azure functions.

All of the gates are queried at a fixed frequency and the deployment waits until all the gates pass in the same interval. Each query of the gate is called a sample and the time between two samples is called the sampling interval.

Initial samples from the monitoring systems may not be accurate as not enough information is available for the new deployment. Establishing a delay before evaluation ensures the release does not progress in this period, even if the initial samples are successful.

The following diagram illustrates the gate evaluation flow where, after the initial stabilization delay period and three subsequent sampling intervals, the deployment is approved.

Conceptual execution of gates

Gates ensure the release waits for you to react to feedback and fix any issues within a timeout period. Until the issues are fixed, the gate samples continue to fail and the deployment waits. Once the issues are fixed, the next sample from the gates becomes successful and the deployment automatically proceeds.

If a new release is required to fix the issues, then you can cancel the deployment, and manually abandon the current release.

Add gates to release definitions

Gates can be added to an environment in the release definition from the pre-deployment conditions or the post-deployment conditions panel.

Multiple gates can be added to the environment conditions to ensure all the inputs are successful for the release. The timeout period and the sampling interval for the gates are configurable. For additional control, you can continue to require manual approvals in addition to using gates.

For example, the following configuration for a two-environment release definition can be performed using gates.

Start by allowing 24 hours for all blocking bugs to be logged, followed by 2 days for resolution of the bugs. Automatically start the deployment to the Canary environment as soon as all blocking bugs are resolved.

Wait for resolution of blocking bugs before starting deployment

After successful deployment to Canary environment, mark the environment as complete when there are no active feedback incidents and no active alerts in Azure monitor.

complete deployment after active alerts and incidents are resolved

Immediately healthy deployments

If the gates are successful in the first evaluation sample, then deployment is immediately approved.

For example, following is the behavior of pre-deployment gates in a release from the definition created above if no blocking bugs are found after 24 hours.

gates succeed on first sample after stabilization

Deployments for which feedback gets resolved

When there is early feedback for a deployment, initial samples from at least one of the gates would be fail. After the team resolves the issues, successful samples from all the gates are received and the deployment would proceed.

Following is the behavior of post-deployment gates in a release from the definition created above if Azure monitor raises alerts that are resolved.

Deployment completes after the alerts are resolved

Unhealthy Deployments

In case the feedback received is not resolved till timeout, then the deployment is rejected.

Following is the behavior of post-deployment gates in a release from the definition created above if Azure monitor raises alerts that are not resolved for 29 hours.

Deployments are rejected if gates are not successful till timeout

Learn More about release approvals and gates

We hope release gates will help your team release applications with higher confidence and fewer manual inputs. We’ll be adding additional controls to increase the efficacy of managing the sampling of gates before the end of preview.

If you encounter any issues or have feedback, please tell us on the Visual Studio Developer Community. We would appreciate hearing what you think.

Category
CI/CDDevOps

Author

Shashank Bansal
Principal Product Manager

I am a product manager on the Azure DevOps and GitHub ecosystem team.

0 comments

Discussion are closed.