Updates to Azure Pipelines Runtime Variables Settings [Updated]

Gloridel Morales

May 19th, 202237 1

We have gotten a lot of feedback on this change and after internal deliberation, we are now rolling back this change ASAP.

Final Update as of 5/19/22 @ 10:08 AM PST:

  • We have now fully rolled back the change that caused the issue. You should no longer experiencing this issue. * We will go back to the drawing board to re-work this fix. Our goal is to provide a high bar of security without breaking our customers.

  • We will also do a postmortem on this change to gain a better understanding of what went wrong. Our goal is to understand what we missed so we avoid rolling out this kind of breaking changes in the future.

Again, I am deeply sorry for the inconvenience and disruption this has caused. We remain deeply committed to making sure our customers have a first-class experience using Azure DevOps. Cheers!

  • Benibo Ajumogobia, Azure DevOps Security PM

Gloridel Morales Senior Program Manager, Azure DevOps

Read next

37 comments

Discussion is closed. Login to edit/delete existing comments.

  • Jonathan Rupp 0

    In the last hour, I’ve started getting this error from my CI build when I push new code to my PRs on Github:

    You can’t set the following variables (system.pullRequest.pullRequestId,system.pullRequest.pullRequestNumber,system.pullRequest.mergedAt,system.pullRequest.sourceBranch,system.pullRequest.targetBranch,system.pullRequest.sourceRepositoryUri,system.pullRequest.sourceCommitId). If you want to be able to set these variables, then edit the pipeline and select Settable at queue time on the variables tab of the pipeline editor.

    Do I have to make adjustments to my CI build definition to mark these system variables as “Settable at queue time”? These system variables don’t appear in the “variables” tab.

    • Benibo AjumogobiaMicrosoft employee 0

      Hi Jonathan, my name is Beni and I am a PM on Azure DevOps. To answer your question: Yes, you do need to make adjustments to your CI build definitions. We believe that should fix the problems as we disabled allowance to override variables that are not defined in pipeline.

      • Karl Parry 0

        EDIT: Microsoft is rolling this back.

        Just to confirm we need to set this for every pipeline we have individually? This means right now we have to add these variables to the pipelines (as they don’t exist at all currently as they are managed by the GitHub Integration not something we set up).

        • Jean-Michel Mugnes 0

          +1

          • Benibo AjumogobiaMicrosoft employee 0

            We are rolling back this change. I am very sorry for the inconvenience.

      • Jonathan Rupp 0

        Can you clarify exactly what change we need to make?

        The “variables tab” in ADO is blank for my .yaml CI build – I added this to the variables block in the .yaml file, but the build still fails with the same error:

          system.pullRequest.pullRequestId: ''
  system.pullRequest.pullRequestNumber: ''
  system.pullRequest.mergedAt: ''
  system.pullRequest.sourceBranch: ''
  system.pullRequest.targetBranch: ''
  system.pullRequest.sourceRepositoryUri: ''
  system.pullRequest.sourceCommitId: ''
        • Karl Parry 0

          EDIT: Microsoft is rolling this back. If they plan to enable this again you can disable this setting at the organisation/project level. Got to Organisation -> [Pipelines Section] Settings -> Toggle Off “Limit variables that can be set at queue time”. You will need to do this at the organisation and project level. Thanks Arthur chambers for this

          I agree this is not good and should be reverted as its an impossible overhead that has been added without considering impacts. Atleast add a lead time so we can work at setting this up ahead of it breaking. Since its the GitHub Integration they broke it feels like this was not tested or considered.

          I got this working by:
          – go to the Pipeline in ADO
          – edit the pipeline in ADO directly
          – click to variables
          – then add all of those variables – make sure you tick “Let users override this value when running this pipeline”. As far as I know there is no way to set this via the YAML files. Microsoft can you confirm?

          • Jean-Michel Mugnes 0

            Again i agree 100% with @Karl Parry. Though i am using bitbucket (so this is not only gitHub integration they broke…)

            However , i found that by recreating a new pipeline pointing to my original Yaml file (no system variables added), i am able to trigger it manually. But it is not triggered anymore by a branch update in a PR (as it did before)

          • Stephen Roughley 0

            Karl!!! Do you have a way I can send you something for a beer or anything? You have saved my neck ❤

          • Benibo AjumogobiaMicrosoft employee 0

            We are rolling back this change. I am very sorry for the inconvenience.

        • Jonathan Rupp 0

          I appear to have gotten this working by manually adding these variables to the variables tab and checking “let users override this value when running this pipeline”. Changing the .yaml doesn’t appear to be necessary.

          https://i.imgur.com/yu1AHFu.png
          https://i.imgur.com/SroARJw.png

  • Jean-Michel Mugnes 0

    i have the exact same issue as @Jonathan Rupp. And no clear way how to fix this either

    • Benibo AjumogobiaMicrosoft employee 0

      Hi Jean-Michel, please see my reply to Jonathan above.

      • Cesar Laforet Coelho 0

        I Have the same problem, and you answer is not clear to me! Everything was working and you broke it supposedly in the name of security… sure, at least give me examples of the code I need to add to my pipelines please.

        • Benibo AjumogobiaMicrosoft employee 0

          We are rolling back this change. I am very sorry for the inconvenience.

        • Vince Bowdren 0

          It isn’t code you need to add; you need to declare the variables in the pipeline definition. You can do this through the azure devops UI (https://dev.azure.com/yourorganisation/yourproject/_build ); drill down to the affected pipeline, select Edit, press Variables, and add a declaration for each variable you will be using.

          • Cesar Laforet Coelho 0

            Thank you Vince, at least, it is a clear explanation, with step by step.
            We are using YAML for a reason, so settings variables one by one in the UI doesn’t make sense for us, and to make it worst we are not using (directly) the affected variables, because they are system variables so setting them makes even less sense.
            If they didn’t revert the change I was expecting some other way by codding it in YAML.

  • Adam Harries 0

    You’ve just made a change that has effectively broken GitHub PR triggers for any team that uses them, with no warning and no workaround.

    • Benibo AjumogobiaMicrosoft employee 0

      We are rolling back this change. I am very sorry for the inconvenience.

  • Stephen Roughley 0

    You have broken our builds!!! We are using YAML builds and we are dealing with a P1 outage and now we are unable to deploy from GitHub. Give us explicit instructions for addressing this issue with YAML pipelines right away.

    • Benibo AjumogobiaMicrosoft employee 0

      We are rolling back this change. I am very sorry for the inconvenience.

  • Geovanny Alzate Sandoval 0

    We’re using YAML pipelines and we are NOT even setting those system variables, how do we fix it? that’s blocking us, we need to make a release.

    Error:

    You can’t set the following variables (system.pullRequest.pullRequestId,system.pullRequest.pullRequestNumber,system.pullRequest.mergedAt,system.pullRequest.sourceBranch,system.pullRequest.targetBranch,system.pullRequest.sourceRepositoryUri,system.pullRequest.sourceCommitId). If you want to be able to set these variables, then edit the pipeline and select Settable at queue time on the variables tab of the pipeline editor.
    • Benibo AjumogobiaMicrosoft employee 0

      We are rolling back this change. I am very sorry for the inconvenience.

  • Arthur Chambers 0

    Workaround for those with issues:
    In Azure Devops Project:
    Project Settings -> [Pipelines Section] Settings -> Toggle Off “Limit variables that can be set at queue time”

    • Benibo AjumogobiaMicrosoft employee 0

      We are rolling back this change. I am very sorry for the inconvenience.

    • Karl Parry 0

      Thanks for this. Now I know this exists at least

    • Geovanny Alzate Sandoval 0

      Thanks Arthur!

    • Jean-Michel Mugnes 0

      alas free tier users cannot toggle this setting….

  • Christopher Crook 0

    Can we be assured that future changes with potential to break CI integrations when default Pipeline Settings are in place will be thoroughly tested in the future?

    This wasn’t a great experience.

    • Benibo AjumogobiaMicrosoft employee 0

      Absolutely! This was a miss on our end and I am deeply sorry for that. We will go back to the drawing board to make on this change. Afterwards, we will do a post mortem on this to figure out what went wrong so we avoid, as best we can, this happening again.

      • Christopher Crook 0

        Might I suggest making sure that changes like this appear somewhere other than buried away in a blog post? Didn’t seem like it showed up in the Feature Timeline or “What’s New” pages here: https://docs.microsoft.com/en-us/azure/devops/release-notes/features-timeline

        This made trying to figure out what was going on more difficult as well.

  • Tovias Rojas, Hermes 0

    How do we know the roll back is done?

    We have a lot of PRs waiting for the build to pass successfully

    • Benibo AjumogobiaMicrosoft employee 0

      The roll back is complete.

  • Markus Szumovski 0

    Is the Azure DevOps Server On-Premises 2020.1.2 impacted by this issue and do we need to wait for a patch for this issue to be fixed?

  • William Charlton 0

    Ms Morales:

    I currently have the 10/6/2020, Azure DevOps Server 2020 RTW installed. ADS admin console shows 18.170.30525.1.

    The release notes for the May 17, 2022, Azure DevOps Server 2020.1.2 says that I can “upgrade from Azure DevOps Server 2020”

    Can I upgrade directly from the 10/6/2020 Azure DevOps Server 2020 RTW to ADS 2020.1.2, or do I first need to install the 3 patches for ADS 2020? (Dec 8, 2020 ADS 2020 Patch 1; Jan 12, 2021 ADS 2020 Patch 2, and Feb 9, 2021 ADS 2020 Patch 3)

Feedback usabilla icon