{"id":6053,"date":"2018-02-15T10:29:24","date_gmt":"2018-02-15T18:29:24","guid":{"rendered":"\/developerblog\/?p=6053"},"modified":"2020-03-14T17:28:06","modified_gmt":"2020-03-15T00:28:06","slug":"copy-custom-vm-images-on-azure","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/ise\/copy-custom-vm-images-on-azure\/","title":{"rendered":"Copy Custom VM Images on Azure"},"content":{"rendered":"

Background<\/h2>\n

We recently worked with Namogoo<\/a>, a cloud security and analytics startup that works with e-commerce and retail companies to protect against\u00a0ads that direct customers to competitors. Virtual machines (VMs) with custom images are a key part of Namogoo’s system.\u00a0 These images include all proprietary software and configurations required for their offerings. The VMs that host these images are used in an elastic manner and auto-scaled as required by traffic patterns.<\/p>\n

Namogoo made the decision to migrate a large portion of their infrastructure from AWS to Azure with support from Microsoft. This code story explains how we created an azure-cli extension that simplifies the process of distributing VM images globally.<\/p>\n

The Challenge<\/h2>\n

The idea behind custom images is that you bring up a VM, do all the installation and configuration needed for the solution based on it to work and finally, package the VM’s disks as an image to be re-used later on. Since the image already includes the specific customization for a solution, using it enables the environment to come online faster compared to using other methods (like using a “stock” image followed by init scripts). This tie factor becomes more important in complex systems that use auto-scaling because when more machines are needed to serve traffic, you want them available sooner rather than later.<\/p>\n

A custom image is a resource in a specific region; as a result, it needs to be replicated in all target regions.<\/p>\n

Replicating images across regions in Azure requires users to:<\/p>\n

    \n
  1. Create disk snapshots from a source image<\/li>\n
  2. Create a storage account in the destination region<\/li>\n
  3. Create a container in the destination storage account<\/li>\n
  4. Copy a snapshot blob to the destination container<\/li>\n
  5. Create a snapshot of the blob\u00a0in the destination region<\/li>\n
  6. Create an image from the snapshot<\/li>\n
  7. Clean up temporary resources<\/li>\n<\/ol>\n

    Steps 2-6 need to be performed across each destination region to which we are copying the image. Furthermore, these steps are just an outline and the actual procedure is between 10 to 15 steps, depending on the scenario. As you can see, the process is not straightforward. Although there are a few automation scripts available on the web, they require some tinkering to get the job done.<\/p>\n

    Namogoo updates their images frequently and deploys them in many different regions and this whole process is tedious and time-consuming on Azure as compared to AWS, which was making it difficult to migrate more workloads to Azure. The objective of our solution was to create a way to copy images between regions<\/strong> that would be simple to use<\/strong>.<\/p>\n

    <\/p>\n

    The Solution<\/h2>\n

    Together with the relevant product groups at Microsoft, we decided to create a new azure-cli\u00a0<\/em>command extension\u00a0that consolidates the process outlined above into a single command.<\/p>\n

    Example of the image copy command:<\/p>\n

    az image copy --source-resource-group mySources-rg --source-object-name myImage --target-location uksouth --target-resource-group \"images-repo-rg\"<\/pre>\n
    A command extension is a relatively\u00a0new feature<\/a>\u00a0in the Azure CLI that offers new functionality not available when the CLI is first installed. This feature is similar to Git subcommands, vscode extensions, etc. that allow users to extend the core application.<\/p>\n
    The list-available command lists publicly available extensions:<\/p>\n
    az extension list-available<\/pre>\n
    A specific extension can be added to your CLI using the add command:<\/p>\n
    az extension add --name myExtension<\/pre>\n
    Developing an extension is very similar to developing other CLI commands and consists of the following components:<\/p>\n
      \n
    1. Parameter definition<\/li>\n
    2. Logic<\/li>\n
    3. Help text<\/li>\n<\/ol>\n
      You can find more information on Azure CLI extensions<\/a>\u00a0on GitHub.<\/p>\n
      Image Copy Extension<\/h2>\n
      Using Internal Commands<\/h3>\n
      Our extension encapsulates the process of copying the image by using existing CLI commands. To keep the extension independent and decoupled from the implementation of commands used, we decided to invoke commands externally (as if they were called from a script).<\/p>\n
      The extension works in steps, where each step goes through the following stages:<\/p>\n
        \n
      1. Prepare the command to run<\/li>\n
      2. Run the command<\/li>\n
      3. Process the result<\/li>\n<\/ol>\n
        For example, the following function creates a new resource group:<\/p>\n
        from azext_imagecopy.cli_utils import run_cli_command, prepare_cli_command\r\n\r\ndef create_resource_group(resource_group_name, location):\r\n    # check if target resource group exists\r\n    cmd = prepare_cli_command(['group', 'exists',\r\n                               '--name', resource_group_name], output_as_json=False)\r\n\r\n    cmd_output = run_cli_command(cmd)\r\n\r\n    if 'false' in cmd_output:\r\n        # create the target resource group\r\n        logger.warn(\"Creating resource group: %s\", resource_group_name)\r\n        cmd = prepare_cli_command(['group', 'create',\r\n                                   '--name', resource_group_name,\r\n                                   '--location', location])\r\n\r\n        run_cli_command(cmd)<\/pre>\n
        Performance<\/h3>\n
        Copying an image from one region to another can take some time since the actual copying is performed asynchronously by Azure storage\u00a0infrastructure on spare bandwidth between the regions. If the regions are close, this process can be relatively quick and only take around 4 minutes. However, it can take considerably longer between far apart regions (for example, copying between Europe and Australia can sometimes take 30 minutes).<\/p>\n
        In cases where it is necessary to copy an image to several regions, it makes sense to do so in parallel to save time and avoid redundant actions on the source image.<\/p>\n
        Using the extension<\/h3>\n
        First, install the extension:<\/p>\n
        az extension add --name image-copy-extension<\/pre>\n
        Then, call it as you would any other az<\/span>\u00a0command:<\/p>\n
        az image copy --source-resource-group mySources-rg --source-object-name myImage --target-location uksouth northeurope --target-resource-group \"images-repo-rg\" --cleanup<\/pre>\n
        The extension piggybacks on the image<\/em> parent command and extends it to add the copy<\/em> functionality. The parameters listed describe the source and destination. The last one, –cleanup<\/span>\u00a0 deletes all temporary resources created in the process like snapshots, storage accounts, etc. In addition, all resources created (temporary and permanent) are tagged by the created_by=image-copy-extension<\/span>\u00a0 tag.<\/p>\n
        While the command is running you can see the various stages as well as the progress of the parallel copy process to each region. Below is an example of such an output:<\/p>\n
        \"\"<\/p>\n
        Source Disk<\/h3>\n
        The whole process of copying an image relies upon the existence of the disk used to create the source image. Currently, there’s no way to access the actual resources behind the source image so we have to use the disk originally used to create the image.<\/p>\n
        Conclusion<\/h2>\n
        Copying custom VM images between regions is\u00a0a\u00a0basic requirement for many Azure deployments; as such, the learning in this code story is reusable in many other scenarios. The image-copy extension has helped to streamline Namogoo’s DevOps processes on Azure. It will also enable other customers to reduce deployment times by providing a\u00a0quick and easy way to copy images across regions in Azure.<\/p>\n
        Adding functionality to azure-cli is simple and extensions can be shared publicly for everyone to use, or privately within your organization. If you try out\u00a0image-copy-extension<\/em><\/strong><\/a>, let us know what you think in the comments below.<\/p>\n
        Resources<\/h3>\n