May 19th, 2022

Updates to Azure Pipelines Runtime Variables Settings [Updated]

Gloridel Morales
Gloridel Morales
Senior Technical Program Manager

We have gotten a lot of feedback on this change and after internal deliberation, we are now rolling back this change ASAP.

Final Update as of 5/19/22 @ 10:08 AM PST:

  • We have now fully rolled back the change that caused the issue. You should no longer experiencing this issue. * We will go back to the drawing board to re-work this fix. Our goal is to provide a high bar of security without breaking our customers.

  • We will also do a postmortem on this change to gain a better understanding of what went wrong. Our goal is to understand what we missed so we avoid rolling out this kind of breaking changes in the future.

Again, I am deeply sorry for the inconvenience and disruption this has caused. We remain deeply committed to making sure our customers have a first-class experience using Azure DevOps. Cheers!

  • Benibo Ajumogobia, Azure DevOps Security PM
Category
Security

Author

Gloridel Morales
Gloridel Morales
Senior Technical Program Manager

Gloridel is a Senior Technical Program Manager on the Azure DevOps team.

37 comments

Discussion is closed. Login to edit/delete existing comments.

Newest
Newest Popular Oldest
  • Daniel StrommenMicrosoft employee

    Hello, I am getting the same error on my classic build pipelines (You can’t set the following variables) and there is no option in the classic build Variable UI to mark them as “settable at queue time”. Did the bad change get rolled out again?

  • William Charlton

    Ms Morales:

    I currently have the 10/6/2020, Azure DevOps Server 2020 RTW installed. ADS admin console shows 18.170.30525.1.

    The release notes for the May 17, 2022, Azure DevOps Server 2020.1.2 says that I can "upgrade from Azure DevOps Server 2020"

    Can I upgrade directly from the 10/6/2020 Azure DevOps Server 2020 RTW to ADS 2020.1.2, or do I first need to install the 3 patches for ADS 2020? (Dec 8, 2020 ADS 2020 Patch 1; Jan 12, 2021...

    Read more
  • Markus Szumovski

    Is the Azure DevOps Server On-Premises 2020.1.2 impacted by this issue and do we need to wait for a patch for this issue to be fixed?

  • Tovias Rojas, Hermes · Edited

    How do we know the roll back is done?

    We have a lot of PRs waiting for the build to pass successfully

  • Christopher Crook

    Can we be assured that future changes with potential to break CI integrations when default Pipeline Settings are in place will be thoroughly tested in the future?

    This wasn’t a great experience.

    • Benibo AjumogobiaMicrosoft employee

      Absolutely! This was a miss on our end and I am deeply sorry for that. We will go back to the drawing board to make on this change. Afterwards, we will do a post mortem on this to figure out what went wrong so we avoid, as best we can, this happening again.

      • Christopher Crook

        Might I suggest making sure that changes like this appear somewhere other than buried away in a blog post? Didn’t seem like it showed up in the Feature Timeline or “What’s New” pages here: https://docs.microsoft.com/en-us/azure/devops/release-notes/features-timeline

        This made trying to figure out what was going on more difficult as well.

  • Arthur Chambers

    Workaround for those with issues:
    In Azure Devops Project:
    Project Settings -> [Pipelines Section] Settings -> Toggle Off “Limit variables that can be set at queue time”

    • Jean-Michel Mugnes

      alas free tier users cannot toggle this setting….

    • Geovanny Alzate Sandoval

      Thanks Arthur!

    • Karl Parry

      Thanks for this. Now I know this exists at least

    • Benibo AjumogobiaMicrosoft employee

      We are rolling back this change. I am very sorry for the inconvenience.

  • Geovanny Alzate Sandoval · Edited

    We're using YAML pipelines and we are NOT even setting those system variables, how do we fix it? that's blocking us, we need to make a release.

    Error:

    <code>

    Read more
    • Benibo AjumogobiaMicrosoft employee

      We are rolling back this change. I am very sorry for the inconvenience.

  • Stephen Roughley

    You have broken our builds!!! We are using YAML builds and we are dealing with a P1 outage and now we are unable to deploy from GitHub. Give us explicit instructions for addressing this issue with YAML pipelines right away.

    • Benibo AjumogobiaMicrosoft employee

      We are rolling back this change. I am very sorry for the inconvenience.

  • Adam Harries

    You’ve just made a change that has effectively broken GitHub PR triggers for any team that uses them, with no warning and no workaround.

    • Benibo AjumogobiaMicrosoft employee

      We are rolling back this change. I am very sorry for the inconvenience.

  • Jean-Michel Mugnes

    i have the exact same issue as @Jonathan Rupp. And no clear way how to fix this either

    • Benibo AjumogobiaMicrosoft employee

      Hi Jean-Michel, please see my reply to Jonathan above.

      • Cesar Laforet Coelho

        I Have the same problem, and you answer is not clear to me! Everything was working and you broke it supposedly in the name of security… sure, at least give me examples of the code I need to add to my pipelines please.

      • Vince Bowdren

        It isn’t code you need to add; you need to declare the variables in the pipeline definition. You can do this through the azure devops UI (https://dev.azure.com/yourorganisation/yourproject/_build ); drill down to the affected pipeline, select Edit, press Variables, and add a declaration for each variable you will be using.

      • Cesar Laforet Coelho

        Thank you Vince, at least, it is a clear explanation, with step by step.
        We are using YAML for a reason, so settings variables one by one in the UI doesn't make sense for us, and to make it worst we are not using (directly) the affected variables, because they are system variables so setting them makes even less sense.
        If they didn't revert the change I was expecting some other way by codding it...

        Read more
      • Benibo AjumogobiaMicrosoft employee

        We are rolling back this change. I am very sorry for the inconvenience.

Read next