December 14th, 2021

Updated: Azure DevOps (and Azure DevOps Server) and the log4j vulnerability

Gloridel Morales
Senior Technical Program Manager

Update: We released patches for Azure DevOps Server and TFS 2018.3.2 to include an upgraded version of Elasticsearch. Check out the blog post for details.

For the most part, Azure DevOps (and Azure DevOps Server) are built on .NET and do not use the Apache log4j library whose vulnerabilities (CVE-2021-44228, CVE-2021-45046, Microsoft security blog post) have been the focus of so much recent attention. The Search feature in both Azure DevOps and Azure DevOps Server does use this library, however, as part of its dependency on Elasticsearch.

Azure DevOps is not vulnerable. Even so, we are taking this opportunity to make additional improvements as part of our overall defense in depth strategy for the hosted service.

For Azure DevOps Server (and older versions of Team Foundation Server):

  • Installations without Search configured are not vulnerable and no action is required.
  • For Azure DevOps Server 2020, Azure DevOps Server 2019, and Team Foundation Server 2018 our recommendation is the same:

  • Upgrade the Java Virtual Machine on the server where the Search feature is installed to the latest release with the same major version (and then restart Elasticsearch).

  • Remove the JndiLookup class from the jar file (and then restart Elasticsearch). Most references to this online give a command that will work only on Linux. MVP Jesse Houwing has helpfully included commands that will work on Windows using 7-zip in his blog post here. They will be similar to: 7z.exe d log4j-core-.jar org/apache/logging/log4j/core/lookup/JndiLookup.class

  • Team Foundation Server 2017 is not vulnerable to CVE-2021-44228 or CVE-2021-45046. It is, however, dependent on a version of Elasticsearch that is end-of-life from a support perspective. We recommend upgrading to a newer version of Team Foundation Server / Azure DevOps Server (documentation) or uninstalling the Search feature (documentation).

We are still exploring generation of patches to simplify this process, and will continue posting updates here as we learn more.

Additionally, the Linux versions of the Azure Pipelines agent include an older version of log4j as part of the Team Explorer Everywhere command line used to interact with Team Foundation Version Control. This version of log4j is not vulnerable to CVE-2021-44228 or CVE-2021-45046. It is end-of-life and includes other vulnerabilities, but we have previously confirmed that these vulnerabilities are not exploitable on Azure Pipelines agents. No action is required for users of Linux Azure Pipelines agents.

Last updated: 1/26/2022 @ 12:06 pm PST

Author

Gloridel Morales
Senior Technical Program Manager

Gloridel is a Senior Technical Program Manager on the Azure DevOps team.

9 comments

Discussion is closed. Login to edit/delete existing comments.

  • Markus

    Hi Gloridel,

    is there already a ETA for the Elastic Search fix?

    • Gloridel MoralesMicrosoft employee Author

      Hi Markus, we plan to release fixes with our next patch release but don’t have a date yet.

      • Oleg Komarov

        Hi Gloridel, do you have any update information about Elastic Search v6.2 fix?

  • Joe Lucas

    Using https://github.com/hillu/local-log4j-vuln-scanner indicates that Azure Dev Ops Server 2020.1.1 is affected in the ElasticSearch components.

    o indicator for vulnerable component found in c:\Program Files\Azure DevOps Server 2020\Search\ES\elasticsearchv6.2\lib\log4j-core-2.9.1.jar (org/apache/logging/log4j/core/net/JndiManager$JndiManagerFactory.class): log4j 2.9.1-2.10.0
    o indicator for vulnerable component found in c:\Program Files\Azure DevOps Server 2020\Search\ES\elasticsearchv6.2\lib\log4j-core-2.9.1.jar (org/apache/logging/log4j/core/pattern/MessagePatternConverter.class): log4j 2.8-2.9
    o indicator for vulnerable component found in c:\Program Files\Azure DevOps Server 2020\Search\ES\elasticsearchv6.2\lib\log4j-core-2.9.1.jar (org/apache/logging/log4j/core/net/JndiManager$1.class): log4j 2.4-2.11.2
    o indicator for vulnerable component found in c:\Program...

    Read more
    • Jesse Houwing

      currently you are on your own there is no patch or hotfix available from Microsoft yet to patch or upgrade Elastic Search.

  • Sebastian Schütze

    Last week we opened a ticket with premier support where we were assured that no Log4j is being used. Was it a miscommunication or lack of knowledge?

    The statement is written in a way that ElasticSearch bein used with Log4J is not so much of a problem. Could you please elaborate on this?

    Would be nice to get an update.

    • Jesse Houwing · Edited

      For Azure DevOps, the cloud service, everything is patched and no problem.

      For Azure DevOps Server 2019 and up with Search enabled, Elastic Search 6.2.4 is deployed. Elastic Search relies on log4net and this version is vulnerable to information disclosure. According to Elastic this version should not be vulnerable to the RCE-part of this bug.

      For Team Foundation Server 2018 update 2 and up and with Search enabled relies on Elastic Search 5.4.1 and by default is...

      Read more
  • Jesse Houwing

    For a more detailed analysis of Azure DevOps Server and Team Foundation Server check my blogpost.