Over the past year, we have been able to update Azure Pipelines hosted images faster than ever before. We have also rolled out new images – Windows Server 2019 with Visual Studio 2019, Ubuntu 18.04, and macOS Mojave 10.14. We have seen a phenomenal growth in the use of hosted pools.
To prepare for another set of exciting updates this year and to better serve the needs of our growing user base, we have to make room – both in terms of Azure capacity and in terms of human support resources – and remove some of the less used, older images. On March 23, 2020, we’ll be removing the following Azure Pipelines hosted images:
- Windows Server 2012R2 with Visual Studio 2015, a.k.a
vs2015-win2012r2orHosted - macOS X High Sierra 10.13, a.k.a
macOS-10.13 - Windows Server Core 1803, a.k.a
win1803
This blog outlines the steps you must take now if you use one of these images.
Schedule changes
Based on user feedback and higher usage of the vs2015-win2012r2 image, we have decided to delay removal of the vs2015-win2012r2 image until March 31, 2020. Pipelines using the Hosted agent pool will be automatically redirected to use the vs2017-win2016 image.
Current Status
As of March 31, 2020 the following Azure Pipelines hosted images have been removed.
- Windows Server 2012R2 with Visual Studio 2015, a.k.a
vs2015-win2012r2orHosted - macOS X High Sierra 10.13, a.k.a
macOS-10.13 - Windows Server Core 1803, a.k.a
win1803
In addition to removing these hosted images, the Hosted agent pool has been redirected to use the vs2017-win2016 image.
Windows Server 2012R2 with Visual Studio 2015 (vs2015-win2012r2 or Hosted)
We have decided to remove the vs2015-win2012r2 image because it has not been updated in several years, lacks various security patches and hotfixes, and less than 5% of our users use it. This includes the agent pool labeled Hosted. While we have decided to remove the vs2015-win2012r2 image, there are two newer versions of Visual Studio images available in Azure Pipelines – vs2017-win2016 and windows-2019.
We understand that some of you still build your older apps using this image. Here are your options to move forward:
- Identify pipelines using the
vs2015-win2012r2image, by navigating tohttps://dev.azure.com/{yourorganization}/{yourproject}/_settings/agentqueues. From there, you will need to check both the Azure Pipelines and Hosted agent pool for pipelines impacted by this change. Please note, in the Azure Pipelines pool, you will need to use the agent specification dropdown to filter forvs2015-win2012r2. In addition, you can use the query job history for retired images script.
The pipeline name is underneath the job (e.g. PartsUnlimited (2) is the pipeline)
- Use one of the newer versions of Visual Studio, if that is an option for you. In many cases, your apps can be migrated to build on a newer version of Visual Studio with minimal effort. The
vs2017-win2016andwindows-2019images are kept up to date. If you use classic pipelines, edit your pipeline and change the image to one of these. If you use YAML pipelines, update your pipeline by editing its YAML file.
jobs:
- job: Windows
pool:
vmImage: 'vs2017-win2016'
steps:
- script: echo hello from Windows
- Run your builds on a self-hosted agent pool. You can use Azure to create new virtual machines in the cloud, and to install Visual Studio 2015. Once created, configure a self-hosted agent pool in your Azure Pipelines organization, and configure each of your pipelines to use the self-hosted agent pool.
macOS High Sierra 10.13 (macOS-10.13)
We have decided to remove the macOS-10.13 image because less than 5% of users use macOS High Sierra 10.13 and there is a later macOS version image – macOS-10.14 available. In addition, we will be adding the latest macOS version – Catalina or macOS-10.15 on February 3, 2020.
This means that versions of Xcode older than 9.4.1, will no longer be supported. If your builds run on an older version of Xcode, please take some time to update to Xcode 10 or later to continue taking advantage of the Hosted macOS images. You can also set up a self-hosted macOS agent pool with your desired tools and configure your pipelines to use that pool.
To identify pipelines using the macOS-10.13 image, navigate to https://dev.azure.com/{yourorganization}/{yourproject}/_settings/agentqueues. From there, you will need to check both the Azure Pipelines and Hosted macOS High Sierra agent pool for pipelines impacted by this change. Please note, in the Azure Pipelines pool, you will need to use the agent specification dropdown to filter for macOS-10.13. In addition, you can use the query job history for retired images script.
The pipeline name is underneath the job (e.g. PartsUnlimited (2) is the pipeline)
If you have specific questions regarding our Hosted macOS pools, please reach out to Nilofer on Twitter at @nilli_minaj
Windows Container (win1803).
The usage of this image is the lowest among our pools – less than 1%. Any builds that target this pool will start failing on March 23, 2020.
To identify pipelines using the win1803 image, navigate to https://dev.azure.com/{yourorganization}/{yourproject}/_settings/agentqueues. From there, you will need to check both the Azure Pipelines and Hosted Windows Container agent pool for pipelines impacted by this change. Please note, in the Azure Pipelines pool, you will need to use the agent specification dropdown to filter for win1803. In addition, you can use the query job history for retired images script.
The pipeline name is underneath the job (e.g. PartsUnlimited (2) is the pipeline)
To mitigate this:
- Run your builds on a self-hosted agent pool. You can use Azure to create new Windows server core virtual machines in the cloud. Once created, configure a self-hosted agent pool in your Azure Pipelines organization, and configure each of your pipelines to use the self-hosted agent pool.
Brownout schedule
During the next couple of weeks, we will conduct a series of brownouts to highlight builds and releases using these soon-to-be-removed images. We will conduct our first brownout on March 9th from 1800-1900 UTC. After that we will continue to fail builds that use these images for up to 3 hours every day at different times. You will see the following error message in your build or release, if you are affected by a brownout:
This job was temporarily blocked because it uses a Microsoft hosted agent () that will be permanently removed on March 23, 2020. See https://aka.ms/blocked-hosted-agent for more information.
During these periods, any builds and releases that use vs2015-win2012r2, macOS-10.13, or win1803 images will fail. After each brownout period, we will revert the change. If your builds and releases fail during these brownout periods and if you see the error message shown above, please update your image to one of the supported images highlighted in this blog post.
Feedback
We’d really like to hear from you regarding what capabilities you’d like to see us add to the Hosted Agents experience in Azure Pipelines. Please contact us by reaching out @AzureDevOps or by creating an issue on our GitHub repository.
Thank you for using Azure Pipelines!
What about Silverlight projects ? Its only available to VS 2015.
Today is April 7th, but you still remind this in DevOps UI. Screenshot: https://ibb.co/GxhJnWL
YES – Please stop showing this banner!
this comment has been deleted.
Azure pipeline failing when switching from vs2015-win2012r2 to vs2017-win2016 and giving error when try to build Angular 1
‘bower’ is not recognized as an internal or external command,
operable program or batch file.
[error]Process completed with exit code 1.
Was this announced anywhere else? I only saw the banner on DevOps.
These brownouts have been incredibly disruptive Microsoft – we are needing to get a Release out today and have planned our changes to our Pipelines to be done tomorrow, but now our builds are being cancelled at a critical moment. Not great.
We had the same issues. Not fun.
Any chances for this removal to be delayed given the current covid-19 crisis? I guess we are not the only ones to be short staffed these days!
Whitney, we are not calling any specific agent pool or containers, such as the "vs2015-win2012r2". We are just calling the generic "Hosted" pool, yet we get the brownout message and all of our builds failed during the experimental window yesterday. Is that expected?
I thought that by following your example and making sure we weren't explicitly calling the set of agents that were about to retire was sufficient. Did we have to update all of our build defs (yaml) to use the "windows-2019" agent pool?
By the way, does the "windows-2019" agent pool contain any version of Visual Studio?
+1 Builds in our ‘Hosted’ pool are failing, and there is no option to change the agent specification. Do we need to move all of our pipelines out of this pool or is this just a bug that will age out?
Hi All,
If your pipeline is listed as using the Hosted agent pool. Select Azure Pipelines from the agent pool drop down menu. Once Azure Pipelines is selected, you should see a new drop down titled agent specification. This will give you a list of images to choose.
The pool “Hosted” is a moniker for vs2015 pool. So, please change your pool from “Hosted” to “Azure Pipelines, and select one of the supported images.
Thanks for the clarification and updating the blog. I didn’t know if “Hosted” would just get automatically upgraded, since there is no selection.
“If you use classic pipelines, edit your pipeline and change the image to one of these.” This is very terse. It would be great to expand this with some screenshots. Imagine someone new to Azure DevOps took over a bunch of old build that are now failing!
We have the same issue.
Same here. We’re getting brown-outs on the Windows Hosted Pool and when I choose Azure Pool with the “windows-2019” agent. Is there something wrong with the brown-out logic?
Hey, wait. What is about SharePoint 2013 development? Other images do not have installed SDK for SharePoint 2013. Build leads to error described here https://github.com/actions/virtual-environments/issues/532
Hello. Does anybody cares about SharePoint 2013 development in a Microsoft company? Your other images doesn’t work with SP 2013! You can’t just shut the single working agent down!
This document doesn’t really tell me how to move my release task from vs2015-win2012r2 to a newer version. All it explains is how to find the offending pipeline, which I already knew because I saw the warning in the logs. I can’t find where to change from vs2105-* to something newer. Please consider updating this document with instructions on how to make the actual change that is needed. Thanks.
I figured it out after some experimentation. I have been using Azure Dev Ops for 4 years now and the tasks were setup quite a while ago, which meant they were using "Hosted" as the pipeline, not the new "Azure Pipelines". Hosted has no options when editing the Agent Job. You have to select "Azure Pipelines" to be able to see the list of pipeline choices. From there it is straight forward to select the proper one.
However, there doesn't seem to be a VS2019 enabled pipeline host. I don't know what "windows 2019" means. I assume it is just a...
+1 It would be helpful to provide step-by-step guidance on how to make our pipeline(s) compliant…thanks!
I don’t know if there is any connection, however after this change, my pipelines are in the status queued “Waiting for a job.” until time out returns and cancels the process. Do you know what may be happening?