Copy Custom VM Images on Azure

Tamir Kamara

February 15th, 20180 0

Background

We recently worked with Namogoo, a cloud security and analytics startup that works with e-commerce and retail companies to protect against ads that direct customers to competitors. Virtual machines (VMs) with custom images are a key part of Namogoo’s system. These images include all proprietary software and configurations required for their offerings. The VMs that host these images are used in an elastic manner and auto-scaled as required by traffic patterns.

Namogoo made the decision to migrate a large portion of their infrastructure from AWS to Azure with support from Microsoft. This code story explains how we created an azure-cli extension that simplifies the process of distributing VM images globally.

The Challenge

The idea behind custom images is that you bring up a VM, do all the installation and configuration needed for the solution based on it to work and finally, package the VM’s disks as an image to be re-used later on. Since the image already includes the specific customization for a solution, using it enables the environment to come online faster compared to using other methods (like using a “stock” image followed by init scripts). This tie factor becomes more important in complex systems that use auto-scaling because when more machines are needed to serve traffic, you want them available sooner rather than later.

A custom image is a resource in a specific region; as a result, it needs to be replicated in all target regions.

Replicating images across regions in Azure requires users to:

Create disk snapshots from a source image
Create a storage account in the destination region
Create a container in the destination storage account
Copy a snapshot blob to the destination container
Create a snapshot of the blob in the destination region
Create an image from the snapshot
Clean up temporary resources

Steps 2-6 need to be performed across each destination region to which we are copying the image. Furthermore, these steps are just an outline and the actual procedure is between 10 to 15 steps, depending on the scenario. As you can see, the process is not straightforward. Although there are a few automation scripts available on the web, they require some tinkering to get the job done.

Namogoo updates their images frequently and deploys them in many different regions and this whole process is tedious and time-consuming on Azure as compared to AWS, which was making it difficult to migrate more workloads to Azure. The objective of our solution was to create a way to copy images between regions that would be simple to use.

The Solution

Together with the relevant product groups at Microsoft, we decided to create a new azure-cli command extension that consolidates the process outlined above into a single command.

Example of the image copy command:

az image copy --source-resource-group mySources-rg --source-object-name myImage --target-location uksouth --target-resource-group "images-repo-rg"

A command extension is a relatively new feature in the Azure CLI that offers new functionality not available when the CLI is first installed. This feature is similar to Git subcommands, vscode extensions, etc. that allow users to extend the core application.

The list-available command lists publicly available extensions:

az extension list-available

A specific extension can be added to your CLI using the add command:

az extension add --name myExtension

Developing an extension is very similar to developing other CLI commands and consists of the following components:

Parameter definition
Logic
Help text

You can find more information on Azure CLI extensions on GitHub.

Image Copy Extension

Using Internal Commands

Our extension encapsulates the process of copying the image by using existing CLI commands. To keep the extension independent and decoupled from the implementation of commands used, we decided to invoke commands externally (as if they were called from a script).

The extension works in steps, where each step goes through the following stages:

Prepare the command to run
Run the command
Process the result

For example, the following function creates a new resource group:

from azext_imagecopy.cli_utils import run_cli_command, prepare_cli_command

def create_resource_group(resource_group_name, location):
    # check if target resource group exists
    cmd = prepare_cli_command(['group', 'exists',
                               '--name', resource_group_name], output_as_json=False)

    cmd_output = run_cli_command(cmd)

    if 'false' in cmd_output:
        # create the target resource group
        logger.warn("Creating resource group: %s", resource_group_name)
        cmd = prepare_cli_command(['group', 'create',
                                   '--name', resource_group_name,
                                   '--location', location])

        run_cli_command(cmd)

Performance

Copying an image from one region to another can take some time since the actual copying is performed asynchronously by Azure storage infrastructure on spare bandwidth between the regions. If the regions are close, this process can be relatively quick and only take around 4 minutes. However, it can take considerably longer between far apart regions (for example, copying between Europe and Australia can sometimes take 30 minutes).

In cases where it is necessary to copy an image to several regions, it makes sense to do so in parallel to save time and avoid redundant actions on the source image.

Using the extension

First, install the extension:

az extension add --name image-copy-extension

Then, call it as you would any other az command:

az image copy --source-resource-group mySources-rg --source-object-name myImage --target-location uksouth northeurope --target-resource-group "images-repo-rg" --cleanup

The extension piggybacks on the image parent command and extends it to add the copy functionality. The parameters listed describe the source and destination. The last one, –cleanup deletes all temporary resources created in the process like snapshots, storage accounts, etc. In addition, all resources created (temporary and permanent) are tagged by the created_by=image-copy-extension tag.

While the command is running you can see the various stages as well as the progress of the parallel copy process to each region. Below is an example of such an output:

Source Disk

The whole process of copying an image relies upon the existence of the disk used to create the source image. Currently, there’s no way to access the actual resources behind the source image so we have to use the disk originally used to create the image.

Conclusion

Copying custom VM images between regions is a basic requirement for many Azure deployments; as such, the learning in this code story is reusable in many other scenarios. The image-copy extension has helped to streamline Namogoo’s DevOps processes on Azure. It will also enable other customers to reduce deployment times by providing a quick and easy way to copy images across regions in Azure.

Adding functionality to azure-cli is simple and extensions can be shared publicly for everyone to use, or privately within your organization. If you try out image-copy-extension, let us know what you think in the comments below.

Resources

Cover image source