Service Blog - Azure DevOps

Postmortem: Azure DevOps Service Outages in October 2018

Earlier this month, Azure DevOps experienced several significant service outages, for which we are deeply sorry. As with every significant live site incident, we have completed a detailed root cause analysis for these. Due to the proximity of these incidents and common underlying causes,

Postmortem: VSTS 4 September 2018

Postmortem – VSTS Outage – 4 September 2018

On Tuesday, 4 September 2018, VSTS (now called Azure DevOps) suffered an extended outage affecting customers with organizations hosted in the South Central US region (one of the 10 regions globally hosting VSTS customers).

Postmortem: Global VSTS availability issues – 22 May 2018

Customer Impact:

On 22 May 2018, Visual Studio Team Services (VSTS) experienced a major incident across multiple regions between 15:00 and 16:55 UTC.  An event in a Western European scale unit of the Team Foundation Service (TFS), caused a chain reaction that sporadically took other TFS scale units offline in other regions. 

Postmortem

Postmortem: Azure DevOps Service Outages in October 2018

Earlier this month, Azure DevOps experienced several significant service outages, for which we are deeply sorry. As with every significant live site incident, we have completed a detailed root cause analysis for these. Due to the proximity of these incidents and common underlying causes,

Postmortem – VS Marketplace outage – 4 September 2018

On Tuesday, 4 September 2018, Visual Studio Marketplace suffered an extended outage affecting most of its customers. Marketplace hosts and serves extensions for the Visual Studio IDE, Visual Studio Code, and Azure DevOps. This was the first instance of the Marketplace service going down completely,

Postmortem: VSTS 4 September 2018

Postmortem – VSTS Outage – 4 September 2018

On Tuesday, 4 September 2018, VSTS (now called Azure DevOps) suffered an extended outage affecting customers with organizations hosted in the South Central US region (one of the 10 regions globally hosting VSTS customers).

Postmortem: Global VSTS availability issues – 22 May 2018

Customer Impact:

On 22 May 2018, Visual Studio Team Services (VSTS) experienced a major incident across multiple regions between 15:00 and 16:55 UTC.  An event in a Western European scale unit of the Team Foundation Service (TFS), caused a chain reaction that sporadically took other TFS scale units offline in other regions. 

Updated and Completed Postmortem: Performance Issues and failures in VSTS West Europe – 7 February 2018

A week ago we posted an incomplete postmortem and are now following up with the completed version. If you want the full story of how we progressed through this incident, start by reading that.  This postmortem will cover the full root cause analysis but it won’t rehash the first part of the investigation.

Preliminary Postmortem: Performance Issues and failures in VSTS West Europe – 7 February 2018

Edit February 26, 2018: We have just posted an updated and complete postmortem here: https://devblogs.microsoft.com/devopsservice/?p=16295

Customer Impact

On 7 February 2018 we had an incident which impacted users in our Western European scale unit.

Postmortem – Intermittent Failures for Visual Studio Team Services on 14 Dec 2017

On 14 December 2017 we began to have a series of incidents with Visual Studio Team Services (VSTS) for several days that had a serious impact on the availability of our service for many customers (incident blogs #1 #2 #3).

Postmortem – Availability issues with Visual Studio Team Services on 6 Dec 2017

On 6 December 2017 we had a global incident with Visual Studio Team Services (VSTS) that had a serious impact on the availability of our service (incident blog here). We apologize for the disruption. Below we describe the cause and the actions we are taking to address the issues.

Postmortem – Availability issues with Visual Studio Team Services on 21 November 2017

On 21 November 2017 we had an incident with Visual Studio Team Services (VSTS) that had a serious impact on the availability of our service for many customers (incident blog here). We apologize for the disruption. Below we describe the cause and the actions we are taking to address the issues.

Postmortem – Availability issues with Visual Studio Team Services on 16 November 2017

On 16 November 2017 we had a global incident with Visual Studio Team Services (VSTS) that had a serious impact on the availability of our service (https://blogs.msdn.microsoft.com/vsoservice/?p=15526). We apologize for the disruption. Below we describe the cause and the actions we are taking to address the issues.

Uncategorized

We’ve Moved! – Introducing Azure DevOps Service Status Portal

Today, we’re happy to introduce Azure DevOps service status portal which helps with real time insights into active service events and provides further details on the event being investigated. This portal replaces our current experience using this Service blog.

Postmortem: Azure DevOps Service Outages in October 2018

Earlier this month, Azure DevOps experienced several significant service outages, for which we are deeply sorry. As with every significant live site incident, we have completed a detailed root cause analysis for these. Due to the proximity of these incidents and common underlying causes,

Postmortem – VS Marketplace outage – 4 September 2018

On Tuesday, 4 September 2018, Visual Studio Marketplace suffered an extended outage affecting most of its customers. Marketplace hosts and serves extensions for the Visual Studio IDE, Visual Studio Code, and Azure DevOps. This was the first instance of the Marketplace service going down completely,

Postmortem: VSTS 4 September 2018

Postmortem – VSTS Outage – 4 September 2018

On Tuesday, 4 September 2018, VSTS (now called Azure DevOps) suffered an extended outage affecting customers with organizations hosted in the South Central US region (one of the 10 regions globally hosting VSTS customers).

Postmortem: Global VSTS availability issues – 22 May 2018

Customer Impact:

On 22 May 2018, Visual Studio Team Services (VSTS) experienced a major incident across multiple regions between 15:00 and 16:55 UTC.  An event in a Western European scale unit of the Team Foundation Service (TFS), caused a chain reaction that sporadically took other TFS scale units offline in other regions. 

Updated and Completed Postmortem: Performance Issues and failures in VSTS West Europe – 7 February 2018

A week ago we posted an incomplete postmortem and are now following up with the completed version. If you want the full story of how we progressed through this incident, start by reading that.  This postmortem will cover the full root cause analysis but it won’t rehash the first part of the investigation.

Preliminary Postmortem: Performance Issues and failures in VSTS West Europe – 7 February 2018

Edit February 26, 2018: We have just posted an updated and complete postmortem here: https://devblogs.microsoft.com/devopsservice/?p=16295

Customer Impact

On 7 February 2018 we had an incident which impacted users in our Western European scale unit.

Postmortem – Intermittent Failures for Visual Studio Team Services on 14 Dec 2017

On 14 December 2017 we began to have a series of incidents with Visual Studio Team Services (VSTS) for several days that had a serious impact on the availability of our service for many customers (incident blogs #1 #2 #3).

Postmortem – Availability issues with Visual Studio Team Services on 6 Dec 2017

On 6 December 2017 we had a global incident with Visual Studio Team Services (VSTS) that had a serious impact on the availability of our service (incident blog here). We apologize for the disruption. Below we describe the cause and the actions we are taking to address the issues.

Postmortem – Availability issues with Visual Studio Team Services on 21 November 2017

On 21 November 2017 we had an incident with Visual Studio Team Services (VSTS) that had a serious impact on the availability of our service for many customers (incident blog here). We apologize for the disruption. Below we describe the cause and the actions we are taking to address the issues.