{"id":61240,"date":"2021-03-23T05:45:19","date_gmt":"2021-03-23T13:45:19","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/devops\/?p=61240"},"modified":"2021-03-20T17:31:42","modified_gmt":"2021-03-21T01:31:42","slug":"on-prem-to-the-cloud-devops-ing-everything-as-code-ep-5","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/devops\/on-prem-to-the-cloud-devops-ing-everything-as-code-ep-5\/","title":{"rendered":"On Prem To The Cloud: DevOps-ing Everything As Code (Ep 5)"},"content":{"rendered":"<p>We&#8217;re back with episode 5 in our On Prem To the Cloud journey where we&#8217;ll take a deeper look at getting our <a href=\"https:\/\/docs.microsoft.com\/azure\/devops\/learn\/what-is-infrastructure-as-code?WT.mc_id=devops-9483-zdeptawa\">Infrastructure as Code<\/a> setup in a <a href=\"https:\/\/docs.microsoft.com\/azure\/devops\/learn\/what-is-continuous-integration?WT.mc_id=devops-9483-zdeptawa\">Continuous Integration<\/a>\/<a href=\"https:\/\/docs.microsoft.com\/azure\/devops\/learn\/what-is-continuous-delivery?WT.mc_id=devops-9483-zdeptawa\">Continuous Delivery<\/a>(CI\/CD) pipeline &#8211; but first, let&#8217;s take a quick look at how we got to this point.<\/p>\n<p>First, <a href=\"https:\/\/twitter.com\/jaydestro\">Jay<\/a> showed Abel how to <a href=\"https:\/\/devblogs.microsoft.com\/devops\/on-prem-to-the-cloud-lift-and-shift-ep-2?WT.mc_id=devops-9483-zdeptawa\">migrate the application environment into Azure<\/a>, next <a href=\"https:\/\/twitter.com\/abelsquidhead\">Abel<\/a> and <a href=\"https:\/\/twitter.com\/damovisa\">Damian<\/a> showed us how to build a CI\/CD pipeline for the <a href=\"https:\/\/www.youtube.com\/watch?v=KEYOsMWA5ak\">application and database<\/a>, and finally <a href=\"https:\/\/twitter.com\/StevenMurawski\">Steven<\/a> took all of our infrastructure requirements and turned them into <a href=\"https:\/\/docs.microsoft.com\/azure\/devops\/learn\/what-is-infrastructure-as-code?WT.mc_id=devops-9483-zdeptawa\">infrastructure as code<\/a> templates and scripts. With this code, we now can reliably and repeatably stand up our entire environment with all necessary requirements and configurations within a matter of minutes. So, what&#8217;s next?<\/p>\n<p>Now that we&#8217;ve got everything that&#8217;s needed to build our entire infrastructure on Azure programmatically, it&#8217;s time to get it setup in an <a href=\"https:\/\/docs.microsoft.com\/azure\/devops\/pipelines\/?WT.mc_id=devops-9483-zdeptawa\">Azure DevOps Pipeline<\/a>. This will allow us to leverage automation we build into our workflow to do everything required for deploying our application. Our workflow will cover everything from building infrastructure and configuring servers to deploying the application for us.<\/p>\n<p><iframe title=\"On Prem To The Cloud: DevOps-ing Everything As Code (episode 5)\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/lImVTGG0wfg?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<h3>Infrastructure as Code is Beautiful<\/h3>\n<p>We&#8217;ve got all this great code from Steven that stands everything up for us &#8211; he&#8217;s given us <a href=\"https:\/\/docs.microsoft.com\/azure\/azure-resource-manager\/templates\/overview?WT.mc_id=devops-9483-zdeptawa\">Azure Resource Manager Templates<\/a>, parameter files that split out variables for us, and awesome PowerShell scripts that will do all the work from start to finish for us. We can run that from the command line, and it works perfectly.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/IAC1.png\" alt=\"Infrastructure Files\" \/><\/p>\n<p>Thinking forward, how can we get these templates and scripts into our workflow? They work great for manual execution, but we know we&#8217;re working within a <a href=\"https:\/\/github.com\">GitHub<\/a> repository and we know we&#8217;ll be collaborating on our infrastructure and application moving forward. Wouldn&#8217;t it be cool if we could take this code and set it up in such a way that when we request changes within our repository, a workflow would automatically trigger the code to run? Let&#8217;s look at doing just that.<\/p>\n<h3>What is an Azure DevOps Pipeline?<\/h3>\n<p>We&#8217;ve decided to use an <a href=\"https:\/\/docs.microsoft.com\/azure\/devops\/pipelines\/?WT.mc_id=devops-9483-zdeptawa\">Azure DevOps Pipeline<\/a> for our workflow automation within our repository. An Azure DevOps Pipeline is a custom CI\/CD workflow that you define based on the needs of your application. It allows you to automatically build, test, deliver, and deploy your projects based on your own custom configurations.<\/p>\n<p>The beauty of building your own custom pipeline is you choose exactly how it&#8217;s configured and exactly how much work it&#8217;s doing. Do you want your pipeline to only kick off tests? You can do that. Do you want it to test your application <em>and<\/em> build a package? It can do that as well. Do you want it to do <em>everything<\/em> including deploy your application to a live environment? Yep, it can do that, too!<\/p>\n<h2>Getting Things Ready<\/h2>\n<p>Ok, so we know what direction we&#8217;re heading. Let&#8217;s make sure we have everything accounted for and that we have a plan on how to execute the configuration of this pipeline.<\/p>\n<h3>Prerequisites<\/h3>\n<p>We&#8217;ve got a few things we are starting off with. They are:<\/p>\n<ul>\n<li>An Azure Subscription<\/li>\n<li>An Azure DevOps Organization and Project<\/li>\n<li>A GitHub repository that holds our Infrastructure as Code and Application<\/li>\n<li>A desire to automate \ud83d\ude42<\/li>\n<\/ul>\n<p>The code Steven gave us does a few things:<\/p>\n<ul>\n<li>Sets up an <a href=\"https:\/\/docs.microsoft.com\/azure\/governance\/policy\/overview?WT.mc_id=devops-9483-zdeptawa\">Azure Policy<\/a> that our resources need to abide by (policy as code)<\/li>\n<li>Builds a <a href=\"https:\/\/docs.microsoft.com\/powershell\/scripting\/dsc\/overview\/overview?WT.mc_id=devops-9483-zdeptawa\">Desired State Configuration (DSC)<\/a> for our servers (configuration as code)<\/li>\n<li>Stands up a Management Resource Group to store our Desired State Configuration (infrastructure as code)<\/li>\n<li>Stands up our Application&#8217;s Resource Group and servers (infrastructure as code)<\/li>\n<li>The main example PowerShell script also deploys the application!<\/li>\n<\/ul>\n<p>It&#8217;s a good idea to take a step back before a project like this to make sure you&#8217;ve got an idea of where you are and where you&#8217;re headed. One step at a time is the best way to tackle workflow creation.<\/p>\n<h2>What We Aren&#8217;t Using<\/h2>\n<p>After a good collaborative discussion with the team (discussions and collaboration are <em>key<\/em> for the DevOps workflow!), we decided there were a couple things included in the code that Steven wrote that we won&#8217;t be using in this pipeline &#8211; the management resource group template and the policy configuration script.<\/p>\n<h3>Management Resource Group<\/h3>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/MGMT.png\" alt=\"Management RG Files\" \/><\/p>\n<p>The management resource group will hold things that will manage our entire organization. We&#8217;ve decided to keep this separate from the actual application&#8217;s resources being it&#8217;s strictly for management; however, we <em>do<\/em> rely on the management resource group to store the Desired State Configuration (DSC) we&#8217;re building. As such, the management resource group deployment needs to happen <em>before<\/em> we run the automation that builds the Desired State Configuration or deploy the application. As such, this resource group automation will be moved outside of this workflow.<\/p>\n<h3>Azure Policy<\/h3>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/POLICY.png\" alt=\"Policy Code\" \/><\/p>\n<p>We have code that will set up a policy to block us from opening our secure RDP port (3389) on our servers. This policy is majorly important for security reasons, but we&#8217;re leaving it out of the infrastructure pipeline &#8211; and here&#8217;s why.<\/p>\n<p>Our Azure Policy sets guardrail for our infrastructure and application. If we attempt to build or configure anything in our pipeline that falls outside of this policy, the policy should disallow it from happening to keep us on track. As such, the workflow we&#8217;re creating to stand up the infrastructure and build the application needs to happen outside of and after the policy configuration.<\/p>\n<p>Our pipeline should not be creating and managing policy that covers a broader scope than the policy resources. This policy is subscription-wide. You can have more focused policies that may make sense to deploy as part of the application\/infrastructure deployment.<\/p>\n<h2>Parts of an Azure DevOps Pipeline<\/h2>\n<p>So, we know what we have and where we&#8217;re going, now it&#8217;s time to look at all the pieces of an Azure DevOps Pipeline we&#8217;ll need to put together to build out our custom workflow.<\/p>\n<h3>Pipeline Variables<\/h3>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/VAR1.png\" alt=\"List of Pipeline Variables\" \/><\/p>\n<p>In our infrastructure as code, Steven made sure to parameterize as much as possible so the variables that may change based on environment would not be hardcoded. This saved a bunch of time in determining what needed to be set to get going with the pipeline. In an Azure DevOps Pipeline, you can set <a href=\"https:\/\/docs.microsoft.com\/azure\/devops\/pipelines\/process\/variables?WT.mc_id=devops-9483-zdeptawa\">variables<\/a> that are essentially key\/value pairs that are specific to that pipeline. Once those variables are set, you can have them override specific parameters and variables within your pipeline configuration as well as in the scripts your pipeline will be running.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/VAR2.png\" alt=\"Secret Variable Example\" \/><\/p>\n<p>One super powerful feature of <a href=\"https:\/\/docs.microsoft.com\/azure\/devops\/pipelines\/process\/variables?WT.mc_id=devops-9483-zdeptawa\">Azure DevOps Pipeline Variables<\/a> is the ability to set a secret variable. These are variables that can be set and used in the pipeline that can no longer be viewed or edited after creation. This allows you to set variables for sensitive things (like personal access tokens, for example) that need to be consumed by your pipeline while simultaneously protecting the actual data of that variable. As a best practice, you should <em>never<\/em> hardcode any passwords or tokens into your application, you should never send them to other coworkers in plain text for use, and you should never push them into a repository. The secret variables in our pipeline help us avoid all those things.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/VAR3.png\" alt=\"Variable Reference\" \/><\/p>\n<p>Pipeline variables also have a handy reference that show exactly how you can refer to them within your code. This can save you some time figuring out the right syntax!<\/p>\n<p>It&#8217;s important to note the variable expression syntax will also indicate when in the pipeline lifecycle the variable will be evaluated. Depending on what you want to happen, this can impact the value provided.<\/p>\n<h3>Triggers<\/h3>\n<p><a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/devops\/pipelines\/build\/triggers?WT.mc_id=devops-9483-zdeptawa\">Triggers<\/a> are what we can setup to run a pipeline workflow automatically. We can define triggers that watch for things like changes to a branch, comments, or pull requests being made that will affect a specific branch.<\/p>\n<h3>Jobs, Tasks, and Stages<\/h3>\n<p>With Azure DevOps Pipelines, you organize your workflow into different stages, jobs, and tasks. How you organize your pipeline is completely up to you and your own personal workflow, but to summarize what each of these is:<\/p>\n<ul>\n<li>\n<p><a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/devops\/pipelines\/process\/phases?WT.mc_id=devops-9483-zdeptawa\">Jobs<\/a> &#8211; every pipeline has at least one job, and a job is just a series of steps that run sequentially as one unit.<\/p>\n<\/li>\n<li>\n<p><a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/devops\/pipelines\/process\/tasks?WT.mc_id=devops-9483-zdeptawa\">Tasks<\/a> &#8211; packaged scripts or procedures that have been abstracted with a set of inputs. You can have tasks that run scripts or perform specific actions that you define. There are <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/devops\/pipelines\/tasks\/?WT.mc_id=devops-9483-zdeptawa\">a number of tasks that are already built-in on Azure DevOps<\/a> as well.<\/p>\n<\/li>\n<li>\n<p><a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/devops\/pipelines\/process\/stages?WT.mc_id=devops-9483-zdeptawa\">Stages<\/a> &#8211; these are the dividers for different steps in a pipeline &#8211; things like &#8220;run some tests&#8221;, &#8220;build some code&#8221;, &#8220;deploy some servers&#8221;. They exist as logical boundaries and allow you to do things like pause your pipeline to add checks and things of that nature.<\/p>\n<\/li>\n<\/ul>\n<h3>Environments<\/h3>\n<p>You might be thinking, &#8220;so we can stand up infrastructure, get it configured, build the app, and have it all ready to deploy&#8230; but how do we target the infrastructure we just created to execute that deployment?&#8221; Well, one option is to use environments in Azure DevOps. An <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/devops\/pipelines\/process\/environments?WT.mc_id=devops-9483-zdeptawa\">Azure DevOps Environment<\/a> is simply a collection of resources (like Kubernetes clusters and virtual machines) that is defined in your pipeline. Think about your typical infrastructure. You most likely have seen &#8216;Development&#8217; (or dev), &#8216;Staging&#8217; (or stage), and &#8216;Production&#8217; (or prod) environments for applications. This is the same way you can use environments &#8211; and in fact, that&#8217;s the way Mercury Health has decided to set up this pipeline workflow.<\/p>\n<h3>So Much More!<\/h3>\n<p>These elements just scrape the surface of what an Azure DevOps Pipeline can use and how it all works, but it&#8217;s plenty for our purposes in getting started. <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/devops\/pipelines\/?WT.mc_id=devops-9483-zdeptawa\">For full documentation on Azure DevOps Pipelines, check out this landing page in Microsoft Docs<\/a>!<\/p>\n<h2>Building the Pipeline<\/h2>\n<p>Ok, we know what we&#8217;re going to build and what we can use to build it&#8230; now what!? Well, we have a few more things we need to think through before we can get started.<\/p>\n<h3>Where Should Our Code Reside?<\/h3>\n<p>The Mercury Health team has been working out of a GitHub repository so far. The team discussed where the code should reside, and we decided to keep the code in GitHub while having our pipeline run in Azure DevOps. What this buys us is really the best of both worlds. We can take advantage of cool GitHub automation features like <a href=\"https:\/\/github.com\/features\/actions\">GitHub Actions<\/a> and cool GitHub security features like <a href=\"https:\/\/codeql.github.com\/docs\/\">CodeQL<\/a> while allowing our actual build and deploy pipeline to exist and run in Azure where our infrastructure will ultimately be stood up.<\/p>\n<h3>Making the Connection Between Azure DevOps and GitHub<\/h3>\n<p>To make this interaction between Azure DevOps and GitHub work, we must create a <a href=\"https:\/\/docs.microsoft.com\/azure\/devops\/pipelines\/library\/service-endpoints?WT.mc_id=devops-9483-zdeptawa\">Service Connection<\/a> between Azure DevOps and GitHub. Don&#8217;t fret though, this is a super easy task.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/SC4.png\" alt=\"First Pipeline\" \/><\/p>\n<p>When we&#8217;re starting with an empty project and we select to create a new pipeline, it&#8217;s going to ask us where our code resides. As you can see, we have the option to choose GitHub for that location.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/SC5.png\" alt=\"Where Is Your Code\" \/><\/p>\n<p>Once we choose GitHub, it&#8217;s going to make the Service Connection for us via interactive authentication. We could alternatively set up a <a href=\"https:\/\/docs.github.com\/en\/github\/authenticating-to-github\/creating-a-personal-access-token\">GitHub Personal Access Token<\/a> if we didn&#8217;t want to rely on user account authentication here, but for the purposes of this initial process, we&#8217;ll go with user authentication to create the Service Connection. When that&#8217;s done, you&#8217;ll be presented with a list of repositories that you have access to as seen below.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/SC6.png\" alt=\"Select Repository\" \/><\/p>\n<p>When you select the appropriate repository, it&#8217;s going to ask you what kind of pipeline you&#8217;d like to setup.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/SC7.png\" alt=\"Configure Pipeline\" \/><\/p>\n<p>We&#8217;re going to begin with a starter pipeline so we can look at the formatting and get an idea of how this pipeline works.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/SC8.png\" alt=\"Review Pipeline\" \/><\/p>\n<p>As you can see at the top of the web editor, this file will save to our repository as <code>azure-pipelines.yml<\/code>.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/SC9.png\" alt=\"Pipeline File Name\" \/><\/p>\n<p>This is the code from our starter pipeline:<\/p>\n<pre><code class=\"yaml\"># Starter pipeline\n# Start with a minimal pipeline that you can customize to build and deploy your code.\n# Add steps that build, run tests, deploy, and more:\n# https:\/\/aka.ms\/yaml\n\ntrigger:\n- master\n\npool:\n  vmImage: ubuntu-latest\n\nsteps:\n- script: echo Hello, world!\n  displayName: 'Run a one-line script'\n\n- script: |\n    echo Add other tasks to build, test, and deploy your project.\n    echo See https:\/\/aka.ms\/yaml\n  displayName: 'Run a multi-line script'\n<\/code><\/pre>\n<p>As you can see, we&#8217;ve got a default <code>trigger<\/code> set to our master branch. This means any changes noticed in that branch will kick off this pipeline. The <code>pool<\/code> section tells our pipeline to use an <code>ubuntu-latest<\/code> host to run these tasks. We&#8217;ll change this later, but for now this is fine being our steps are only running <code>echo<\/code> commands. You can also see this starter pipeline is running <code>script<\/code> tasks.<\/p>\n<p>We can save and run this pipeline if we&#8217;d like &#8211; all it&#8217;s doing is echoing some output for us. When we save this, we&#8217;ll have a pipeline that is running when changes occur in our master branch. We obviously won&#8217;t keep this pipeline like this, but this is a quick way to get a pipeline setup to start iterating on. That&#8217;s all there is to it to get our pipeline file into our repository and setup for changes!<\/p>\n<h3>Making the Connection Between Azure DevOps and Our Azure Subscription<\/h3>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/CON1.png\" alt=\"Service Connections\" \/><\/p>\n<p>This pipeline is going to be interacting with our subscription in a couple ways. It&#8217;s going to be pushing a file into a management resource group that is owned by our subscription, and it&#8217;s also going to be standing up a new resource group with resources in it. As such, we need to make sure Azure DevOps has the rights to do that or this will all fail. Let&#8217;s set up a Service Connection from Azure DevOps to our Azure Subscription!<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/CON2.png\" alt=\"GitHub Service Connection\" \/><\/p>\n<p>Inside our Project Settings under Service Connection, we can see the GitHub connection we just created in the last step. To set one up for our Azure Subscription, we simply need to click on <code>New service connection<\/code> at the top right.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/CON3.png\" alt=\"New Service Connection\" \/><\/p>\n<p>Now we can select the Service Connection type we want. In this case, we&#8217;re going to select <code>Azure Resource Manager<\/code> and click <code>Next<\/code>.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/CON4.png\" alt=\"Service Principal for Service Connection\" \/><\/p>\n<p>We&#8217;ll leave the default here of <code>Service principal (automatic)<\/code> and click <code>Next<\/code>.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/CON5.png\" alt=\"Subscription Level\" \/><\/p>\n<p>We&#8217;ll leave the <code>Scope level<\/code> set to <code>Subscription<\/code>, we&#8217;ll select the appropriate subscription from the drop down, we&#8217;ll name the connection name, and we can give it a description if we&#8217;d like to. Then we&#8217;ll click <code>Save<\/code>.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/CON6.png\" alt=\"Azure and GitHub Service Connections\" \/><\/p>\n<p>Azure DevOps will setup our Service Connection for us via a <a href=\"https:\/\/docs.microsoft.com\/azure\/active-directory\/develop\/app-objects-and-service-principals?WT.mc_id=devops-9483-zdeptawa\">Service Principal<\/a> that will be created in Azure. Pretty easy, right?<\/p>\n<h3>Planning the Stages, Tasks, and Jobs<\/h3>\n<p>Now we&#8217;ve got a pipeline running in our repository as a placeholder, but it&#8217;s not doing anything meaningful regarding building our infrastructure. We need to think about what we need to accomplish with our pipeline and what our first pipeline should look like.<\/p>\n<p>There are many things we want and could add to this pipeline, but we need to focus on getting the pipeline up and going with the least number of requirements &#8211; otherwise we might never launch it. Once the pipeline is up and running, we can always iterate on future versions and add more features and tasks.<\/p>\n<h3>First Iteration of the Pipeline<\/h3>\n<p>Let&#8217;s think back to what the code does that Steven gave us. We won&#8217;t be dealing with the policy or management resource group in this pipeline, but the bits we will be dealing with do the following:<\/p>\n<ul>\n<li>Stands up a resource group and infrastructure for our application.<\/li>\n<li>Builds a Desired State Configuration for our servers.<\/li>\n<li>Builds the application.<\/li>\n<\/ul>\n<p>So, we&#8217;re going to need tasks to do these things for us as well as a task to deploy the application to our environment. Speaking of environment, let&#8217;s get one of those stood up!<\/p>\n<h3>Building the Environment in Azure DevOps<\/h3>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/ENV1.png\" alt=\"Create First Environment\" \/><\/p>\n<p>Creating an Azure DevOps Pipeline Environment is super easy.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/ENV2.png\" alt=\"New Environment\" \/><\/p>\n<p>Nothing is required except the name and a description. We could assign resources to this environment ourselves, but luckily the Desired State Configuration Steven gave us will assign them for us based on the name we choose using an <a href=\"https:\/\/docs.microsoft.com\/en-us\/azure\/devops\/organizations\/accounts\/use-personal-access-tokens-to-authenticate?WT.mc_id=devops-9483-zdeptawa\">Azure DevOps Personal Access Token<\/a> &#8211; so we&#8217;ll just leave our <code>Resource<\/code> set to <code>None<\/code> for now.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/ENV3.png\" alt=\"Environment Get Started\" \/><\/p>\n<h3>Our First Changes to the Pipeline<\/h3>\n<p>Ok, we know where our code lives (GitHub), we have a connection there from Azure DevOps, and we have a starter pipeline with an environment that we can configure. Let&#8217;s start editing the pipeline!<\/p>\n<p>We&#8217;re using a standard Pull Request workflow and are working out of branches in our GitHub repository, so let&#8217;s change our trigger to kick off for the branch I&#8217;m currently working in called <code>episode_5<\/code>. Let&#8217;s also add a path to exclude so our pipeline will ignore any changes we make to our <code>.github<\/code> folder.<\/p>\n<pre><code class=\"yaml\"># Trigger build only on the appropriate branch.\ntrigger:\n  branches:\n    include:\n      - episode_5\n  paths:\n    exclude:\n      - .github\n<\/code><\/pre>\n<p>That looks good. Now, let&#8217;s change the host pool that we&#8217;re using to reflect a Windows host being we&#8217;re going to be working with PowerShell. This isn&#8217;t technically necessary, but let&#8217;s change it anyway &#8211; it won&#8217;t hurt anything for our pipeline!<\/p>\n<pre><code class=\"yaml\"># Setting the pool to run our build on a Windows host.\npool:\n  vmImage: 'windows-latest'\n<\/code><\/pre>\n<p>Awesome! Now our pipeline will use a Windows host instead of an Ubuntu host. It&#8217;s time to look at adding our first real stage!<\/p>\n<h3>Stage One: Building the Application<\/h3>\n<p>It&#8217;s time &#8211; let&#8217;s setup our first stage! You might be thinking we should stand up the infrastructure first, but we&#8217;re going to work on building the application. If you think about it, we really have four things we need to accomplish &#8211; we need to build the Desired State Configuration, stand up the infrastructure, build the application, and deploy the application onto our infrastructure.<\/p>\n<p>Building the Desired State Configuration <em>has<\/em> to happen before we stand up the infrastructure as that Desired State Configuration will be required to configure the servers, and building the application <em>has<\/em> to happen before we can deploy the application to the environment &#8211; but what if we setup the pipeline in this order and something is amiss with the application build? That means we&#8217;ll have to wait for the Desired State Configuration build to run and for the infrastructure task to run before we find out the application failed to build&#8230; and if the application fails to build, then we&#8217;ve just wasted time waiting for it. Let&#8217;s build the application first to make sure it&#8217;s good to go before we spend the time standing up and configuring the infrastructure!<\/p>\n<pre><code class=\"yaml\">stages:\n  # Build and prepare the application \n  # Do this first - if the app doesn't build, we should fail immediately.\n  - stage: PrepareDevApplication\n    jobs:\n    - job: BuildAndPublishApp\n      steps:\n      - task: NuGetToolInstaller@1\n      - task: NuGetCommand@2\n        inputs:\n          restoreSolution: '**\/*.sln'\n      - task: VSBuild@1\n        inputs:\n          solution: '**\/*.sln'\n          msbuildArgs: '\/p:DeployOnBuild=true \/p:WebPublishMethod=Package \/p:PackageAsSingleFile=true \/p:SkipInvalidConfigurations=true \/p:PackageLocation=\"$(build.artifactStagingDirectory)\"'\n          platform: 'Any CPU'\n          configuration: 'Release'\n      - task: CopyFiles@2\n        inputs:\n          SourceFolder: '$(build.artifactStagingDirectory)'\n          Contents: '**\/MercuryHealth.Web.zip'\n          TargetFolder: '$(build.artifactStagingDirectory)\/artifacts'\n      - task: CopyFiles@2\n        inputs:\n          SourceFolder: '$(Build.SourcesDirectory)'\n          Contents: '**\/*.dacpac'\n          flattenFolders: true\n          TargetFolder: '$(build.artifactStagingDirectory)\/artifacts'\n      - task: PublishPipelineArtifact@1\n        inputs:\n          targetPath: '$(build.artifactStagingDirectory)\/artifacts'\n          artifact: 'artifacts'\n          publishLocation: 'pipeline'\n<\/code><\/pre>\n<p>As you can see, our first stage is called <code>PrepareDevApplication<\/code>. Inside this stage, we have one job called <code>BuildAndPublishApp<\/code> that has six tasks to perform. These are the tasks necessary to build our application and put it in our <code>$(build.artifactStagingDirectory)<\/code> under <code>\/artifacts<\/code> where we can consume it later.<\/p>\n<p>Building of these stages takes trial and error &#8211; so we&#8217;ve had to make changes and run the pipeline a few times until it&#8217;s just how we want it. The result is what you see!<\/p>\n<h2>Stage Two: Building the Desired State Configuration (DSC)<\/h2>\n<p>Alright, we have stage one done. Our pipeline is building our application making it ready for deployment. The next stage will build our Desired State Configuration and push it into a storage account within the management resource group that we talked about so we can consume it later via a URI in our Resource Group template.<\/p>\n<pre><code class=\"yaml\">  # Build and prepare the Desired State Configuration (DSC) \n  # This is the server configuration as code and will ensure our servers are configured as intended.\n  - stage: PrepareDesiredStateConfiguration\n    dependsOn: PrepareDevApplication\n    jobs:\n      - job: BuildAndPublishDSC\n        steps:\n        - task: AzurePowerShell@5\n          inputs:\n            azureSubscription: 'on-prem-to-the-cloud(a1b23456-9abc-432d-a1b2-a1bc23d45678)'\n            ScriptType: 'FilePath'\n            ScriptPath: 'Infrastructure\/taskBuildDsc.ps1'\n            azurePowerShellVersion: 'LatestVersion'\n<\/code><\/pre>\n<p>As you can see, this stage is called <code>PrepareDesiredStateConfiguration<\/code> and it depends on the <code>PrepareDevApplication<\/code> stage. What this means is this stage will not run unless the stage it depends on finishes. This ensures my stages are running in the order I need them to.<\/p>\n<p>In this stage, I&#8217;m using the <code>AzurePowerShell@5<\/code> task to run a script that I have stored inside our repository. That script is a snippet I pulled out of the PowerShell script that Steven gave me. Here are the contents of that <code>taskBuildDsc.ps1<\/code> file:<\/p>\n<pre><code class=\"powershell\">$ConfigurationPath = '.\/Infrastructure\/MercHealthConfig.ps1'\n\nWrite-Host \"\"; Write-Host 'Getting the xWebAdministration module to package as part of the published DSC configuration.'\nif (-not (Get-Module -ListAvailable xWebAdministration)) {\n    Install-Module xWebAdministration -RequiredVersion 3.2.0 -Scope CurrentUser -Force\n}\nWrite-Host \"\"; Write-Host 'Getting the xPSDesiredStateConfiguration module to package as part of the published DSC configuration.'\nif (-not (Get-Module -ListAvailable xPSDesiredStateConfiguration)) {\n    Install-Module xPSDesiredStateConfiguration -RequiredVersion 9.1.0 -Scope CurrentUser -Force\n}\nWrite-Host \"\"; Write-Host 'Packaging and publishing the DSC configuration and supporting modules.'\n$Parameters = @{\n    ResourceGroupName  = ${env:MGMTRESOURCEGROUP}\n    ConfigurationPath  = $ConfigurationPath\n    StorageAccountName = ${env:STORAGEACCOUNTNAME}\n    ContainerName      = ${env:STORAGECONTAINERNAME}\n    Force              = $true\n}\nPublish-AzVMDscConfiguration @Parameters | Out-Null\n<\/code><\/pre>\n<p>This script is using another file within the repository called <code>MercHealthConfig.ps1<\/code> that holds all necessary configuration parameters for our servers to build the Desired State Configuration.<\/p>\n<p>You may also notice that we&#8217;re referring to some environment variables in our <code>$Parameters<\/code> &#8211; <code>${env:MGMTRESOURCEGROUP}<\/code>, <code>${env:STORAGEACCOUNTNAME}<\/code>, and <code>{env:STORAGECONTAINERNAME}<\/code>. Those were set in our pipeline variables as mentioned earlier as shown here:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/STG3-1.png\" alt=\"Management RG Variables\" \/><\/p>\n<h2>Stage Three: Building the Resource Group and Infrastructure<\/h2>\n<p>Alright, we&#8217;re halfway there! Let&#8217;s look at stage three &#8211; building the resource group and infrastructure. If you recall from earlier in this post, Steven gave us an Azure Resource Manager template that stands up our Resource Group and everything we need within it for our application. Here is the code I setup in our pipeline to make use of this template:<\/p>\n<pre><code class=\"yaml\">  # Build and prepare the resource group\n  # This is the Azure Resource Group where our infrastructure will be deployed.\n  - stage: PrepareDevResourceGroup\n    dependsOn: PrepareDesiredStateConfiguration\n    jobs:\n    - job: BuildResourceGroup     \n      steps:\n      - task: AzureResourceGroupDeployment@2\n        inputs: \n          azureSubscription: 'on-prem-to-the-cloud(a1b23456-9abc-432d-a1b2-a1bc23d45678)'\n          action: 'Create Or Update Resource Group'\n          resourceGroupName: 'MercuryHealth-dev'\n          location: 'East US 2'\n          templateLocation: 'URL of the file'\n          csmFileLink: 'https:\/\/raw.githubusercontent.com\/zdeptawa\/OnPremToTheCloud\/episode_5\/Infrastructure\/ApplicationRG.json'\n          overrideParameters: '-dscBlobStorageUri $(dscBlobStorageUri) -azureDevOpsToken $(azureDevOpsToken) -domainNameLabel $(domainNameLabel) -vmOsDiskStorageName $(vmOsDiskStorageName) -vmname $(vmname) -username $(username) -password $(password) -azureDevOpsEnvironmentName $(azureDevOpsEnvironmentName) -azureDevOpsProject $(azureDevOpsProject) -azureDevOpsUrl $(azureDevOpsUrl)'\n          deploymentMode: 'Complete'\n<\/code><\/pre>\n<p>This stage is called <code>PrepareDevResourceGroup<\/code> and it depends on the stage before it as well. We&#8217;re using the <code>AzureResourceGroupDeployment@2<\/code> built-in task and setting some inputs. You can see we&#8217;re giving the actual URL for the template. We&#8217;re also overriding some parameters that Steven setup with some Pipeline Variables we set earlier. That&#8217;s happening via <code>overrideParameters<\/code>. That&#8217;s all there is to it for the Resource Group and infrastructure deployment for this application.<\/p>\n<h2>Stage Four: Deploying the Application<\/h2>\n<p>We&#8217;re finally there &#8211; the last stage! We&#8217;ve got our application code built, we built our Desired State Configuration, we stood up our infrastructure&#8230; it&#8217;s time to deploy the application! Let&#8217;s look at this final stage.<\/p>\n<pre><code class=\"yaml\">  # Deploy the application\n  # This is the final step - deploying the application we built in the first step of this pipeline to our infrastructure.\n  - stage: DeployDevApplication\n    dependsOn: PrepareDevResourceGroup\n    variables:\n      - name: websiteName\n        value: 'Mercury Health'\n      - name: databaseName\n        value: MercuryHealthDB\n    jobs:\n      - deployment: DeployTestApplication\n        environment: \n          name: Development\n          resourceType: VirtualMachine\n          tags: mercuryweb\n        strategy:\n          runOnce:\n            deploy:\n              steps:\n              - template: deploy-template.yml\n<\/code><\/pre>\n<p>This stage is named <code>DeployDevApplication<\/code> and it again depends on the stage before it. You can see we&#8217;re setting some variables in-line. We&#8217;re also naming the <code>Development<\/code> deployment environment to deploy into. That&#8217;s telling this deployment stage to push this application out using the <code>deploy-template.yml<\/code> file into our <code>Development<\/code> environment that we stood up earlier that was configured via Steven&#8217;s Desired State Configuration. Awesome how this all ties together, right!?<\/p>\n<h3>Looking at the Pipeline Runs<\/h3>\n<p>We didn&#8217;t go through each run of the pipeline in this blog post as I built it&#8230; why? Because there were <em>a lot<\/em> of them and that would have made this already long blog post way too long. I shrank this screen down to try and capture all the runs, and I still couldn&#8217;t grab them all:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/RUNS1.png\" alt=\"All Runs\" \/><\/p>\n<p>The point here is building a pipeline like this takes time. Every pipeline is different. It takes trial and error. You&#8217;re going to hit walls that hang you up. You&#8217;re going to have to rewrite tasks and jobs. You&#8217;ll have to shuffle code. It&#8217;s just all part of the process &#8211; and you get faster at it every time you do it!<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/RUNS2.png\" alt=\"Run Logs\" \/><\/p>\n<p>The good news is you can easily drill into each of these runs and watch them happen in real time. If you missed the run, no worries &#8211; you can go back and look at all the output from the run to see exactly what happened. This is immensely helpful in debugging what went wrong (or right) in your stages and tasks as you iterate on your pipeline.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/RUNS3.png\" alt=\"Successful Runs\" \/><\/p>\n<p>In the end, you&#8217;ll have a reliable, repeatable, automated process that watches for changes in your repository and fires off a whole host of tasks that you define without you having to think twice about it &#8211; and that&#8217;s super powerful!<\/p>\n<h2>Next Steps for the Pipeline<\/h2>\n<p>We started with Infrastructure as code, and we ended with a Development environment and application that is deployed automatically in our Azure Subscription based off a branch trigger in our GitHub repository via an Azure DevOps Pipeline. That&#8217;s powerful stuff, but what are some next steps we could look at to make it even more powerful in the short term?<\/p>\n<h3>Testing<\/h3>\n<p>Testing is extremely important. When you write tests for your application and codebase, you are literally defining success. When changes are made to your codebase, you can kick off unit tests to verify the integrity of your code changes to make sure that your changes didn&#8217;t introduce anything that waivers from your definition of success that is built into those tests. Some types of tests &#8211; like unit tests &#8211; can kick off and run very rapidly. This is super helpful in failing quickly on a build if something is wrong that is noticed by your tests. The faster you fail a bad build, the less you must wait for the pipeline run, and the faster you can course correct!<\/p>\n<h3>Adding More Environments<\/h3>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/MOREENV.png\" alt=\"More Environments\" \/><\/p>\n<p>We built our pipeline for our Development environment to start, but what about long term? We&#8217;ll want to look at adding some more environments for Staging and Production and any other tiers to our environment as necessary. We can then expand our pipeline to push out to Development, then to Staging, then to Production &#8211; all based off the same trigger and pipeline run!<\/p>\n<h3>Approvals<\/h3>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/APPROVALS1.png\" alt=\"Approvals and Checks\" \/><\/p>\n<p>It&#8217;s powerful to be able to push your changes out across several environments via one pipeline, but that&#8217;s also somewhat dangerous if you don&#8217;t add some checks into the pipeline. One of the most common ways to do this is by adding <a href=\"https:\/\/docs.microsoft.com\/azure\/devops\/pipelines\/process\/approvals?WT.mc_id=devops-9483-zdeptawa\">approvals<\/a> to your Azure DevOps Pipeline Environments.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2021\/03\/APPROVALS2.png\" alt=\"Add Your First Check\" \/><\/p>\n<p>These approvals allow you to halt your pipeline to require checks and approvals before continuing based on environment. This is a powerful feature that&#8217;s built right into Azure DevOps Pipelines.<\/p>\n<h2>We Did It!<\/h2>\n<p>We have a fully working Azure DevOps Pipeline. Now we can look forward to future sprints where we iterate and grow our task list and environments within the pipeline.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We&#8217;re back with episode 5 in our On Prem To the Cloud journey where we&#8217;ll take a deeper look at getting our Infrastructure as Code setup in a Continuous Integration\/Continuous Delivery (CI\/CD) pipeline.<\/p>\n","protected":false},"author":39342,"featured_media":61324,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[224,226],"tags":[],"class_list":["post-61240","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure","category-ci"],"acf":[],"blog_post_summary":"<p>We&#8217;re back with episode 5 in our On Prem To the Cloud journey where we&#8217;ll take a deeper look at getting our Infrastructure as Code setup in a Continuous Integration\/Continuous Delivery (CI\/CD) pipeline.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts\/61240","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/users\/39342"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/comments?post=61240"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts\/61240\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/media\/61324"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/media?parent=61240"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/categories?post=61240"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/tags?post=61240"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}