Deprecating weak cryptographic standards (TLS 1.0 and 1.1) in Azure DevOps Services

Mark Graham

Note: Troubleshooting section added below
Azure DevOps team needed to partially rollback the previous release of TLS 1.0/1.1 deprecation that was run on Jan 31st, 2022. This was due to unexpected issues caused by the change. Here’s a link to the previous blog post related to that release.

TLS 1.0/1.1 deprecation applies to all HTTPS connections to Azure DevOps Services including web API, and git connections to https://dev.azure.com/orgname and https://orgname.visualstudio.com. This does not apply and will not impact the Self-Hosted product: Azure DevOps Server.

Currently, connections to IPv6 endpoints of our services are already on enforced TLS 1.2 so the deprecation is targeting IPv4 endpoints only.

We anticipate minimal impacts to our customers as more than 99.5% of connections made to Azure DevOps Services already use TLS 1.2. Clients have TLS 1.2-compatibility issues because of obsolete OS version or if available updates are not applied (applies for all Windows, macOS and Linux) or legacy .NET Framework installation or OS configuration prohibiting certain TLS cipher suites.

Important Dates

To help our customers identify Azure DevOps-facing TLS 1.2-incompatible software in their systems, we will temporarily disable support for TLS 1.0/1.1 for 12 hours on:

  • March 22, 2022 09:00 to 21:00 UTC for https://orgname.visualstudio.com. TLS 1.0/1.1 connections to Azure DevOps Services using https://orgname.visualstudio.com will temporarily fail to connect. Connections using https://dev.azure.com on TLS 1.0/1.1 will not be impacted.

  • March 24, 2022 09:00 to 21:00 UTC for https://dev.azure.com/orgname. TLS 1.0/1.1 connections to Azure DevOps Services using https://dev.azure.com/orgname will temporarily fail to connect. Connections using https://orgname.visualstudio.com on TLS 1.0/1.1 will not be impacted.

We will be starting permanent rollout from March 31, 2022

How to enable TLS 1.2?

As a key area of focus we recommend Self-Hosted Agents and tooling that calls Azure DevOps Services APIs.

Check these red flags first when looking for TLS 1.2-incompatible flows: – Windows Server 2012 R2 or lower used – .NET Framework 4.7+ not installed on computer where PowerShell or .NET applications call Azure DevOps API

It is key that on the OS one of the four TLS 1.2 cipher suites below is enabled:

  • TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  • TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  • TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (*)
  • TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (*)

NOTE: (*) Cipher suites supported by Windows Server 2012 R2

Quick check of TLS 1.2 compatibility

Following checks are based on querying site status.dev.azure.com. For this domain, the TLS 1.0/1.1 protocols have already been deprecated. Note that checking accessibility of this domain by browser instead of PowerShell is not equivalent. Browsers often use crypto libraries (such as OpenSSL) and thus circumvent the classic HTTP/TLS stack that other software uses.

Manual PowerShell

Run this code in PowerShell console:

(Invoke-WebRequest -Uri status.dev.azure.com -UseBasicParsing).StatusDescription

You have the problem if the above fails with “The underlying connection was closed: An unexpected error occurred on a send.”

YAML

To check your pipeline agents: create testing pipeline OR add this step to existing pipelines:

steps:
  - task: AzurePowerShell@5
    inputs:
      ScriptType: 'InlineScript'
      Inline: '"TLS 1.2 readiness check:"; (Invoke-WebRequest -Uri status.dev.azure.com -UseBasicParsing).StatusDescription'
      FailOnStandardError: true

Pipeline task

This shows a release pipeline task that can be added to perform a check for TLS 1.2 readiness on your Self-Hosted Agents. If the check fails, the task will throw an error that can be actioned.

Pipeline Task

Troubleshooting

For Windows OS, TLS 1.2 is natively supported by all versions from Windows 7 / Windows Server 2008 SP2. However, even at TLS 1.2-compatible OS, issues may be caused by misconfigurations such as when all cipher suites accepted by Azure DevOps are disabled. This may be set up locally or via domain Group Policies.

At application level, capabilities provided by the OS may be narrowed down making the app to use less secure TLS setting. For example, .NET apps which are built against legacy .NET Framework versions may require certain measures to be taken to ensure that TLS 1.2 is favoured.

Based on learnings from supporting our customers, we’ve put together a troubleshooting script which performs TLS 1.2 probe, identifies most common misconfigurations and provides mitigation advice.

You can find the script on GitHub:

https://github.com/microsoft/azure-devops-tls12

The project is work in progress; feel free to contribute with PR or open an issue.

Conclusion

We apologize for any disruption this may cause and appreciate your support to improve our security posture. As always, if you have any questions or concerns related to this announcement, please do not hesitate to reach out to us on Developer Community or by posting your comments below.