Intra-account container copy jobs for Core (SQL) and Cassandra API accounts (Preview)

Announcing the preview for Intra-account container copy jobs. This functionality helps create offline copies of containers for Azure Cosmos DB for both Core (SQL) API and Cassandra API using Azure CLI.

When to use

The container copy feature for Azure Cosmos DB is helpful for the following scenarios:

• Copy all items from one container to another.
• Change the granularity at which throughput is provisioned – from database to container and vice-versa.
• Change the partition key of a container.
• Update the unique keys for a container.
• Rename a container/database.
• Use features that require a new container to be created.

Steps to use intra-account container copy

Here are the steps to use intra-account container copy with an example.

In this example we will copy data from ‘container1’ to ‘container2’ for a Core (SQL) API account named, ‘contoso-container-copydemo’. In this scenario we’re migrating from a contain in a database with shared database throughput ‘db1’ to a container with dedicated throughput in ‘db2’.

1. Start by creating the destination container ‘container2’ with the right settings. In this case, the destination container will have dedicated throughput of 20k RU/s.

2. Stop any ongoing operations on the source container ‘container1’. This is necessary since we do not support online container copy right now. Updates or deletes executed on the source container after the copy job has started may not be captured and copied to the target container. Hence, continuing to perform operations on the source container while the container job is in progress may result in additional or missing data on the target container.

3. Create the container copy job from Cloud Shell Bash. (If you have not used Cloud Shell before, here’s how you open it.)

Alternatively, you may run the copy container jobs from your local machine with Azure CLI installed.

a. First set the shell variables that the container copy command will use:

rg=contoso-container-copydemo
cosmosdbaccountname=contoso-container-copy-sqldemo
copyJobName=copy
sourceDb=db1
sourceContainer=container1
destinationDb=db2
destinationContainer=container2

b. Post that, run the command to create the container copy job:

az cosmosdb dts copy -g $rg --account-name $cosmosdbaccountname --job-name $copyJobName \
--source-sql-container database=$sourceDb container=$sourceContainer \
--dest-sql-container database=$destinationDb container=$destinationContainer

You will see a response like below, indicating that the job has been queued and is in ‘Pending’ state.

The equivalent command for creating a container copy job within a Cassandra API account would be:

az cosmosdb dts copy -g $rg --account-name $cosmosdbaccountname --job-name $copyJobName \
--source-cassandra-table keyspace=$sourceKeySpace table=$sourceTable \
--dest-cassandra-table keyspace=$destinationKeySpace table=$destinationTable

4. Monitor the progress of the job and wait until the job status shows up as ‘Completed’. During the job you can monitor the number of items copied by watching processedCount and the total items to be copied for the job as totalCount using the command below.

az cosmosdb dts show -g $rg --account-name $cosmosdbaccountname --job-name $copyJobName

5. The last step would be to resume the operations by pointing the application or client to the source or destination container as intended.

In this case, we would continue using the destination container ‘container2’.

Next Steps

To get started using container copy jobs, register for the “Intra-account offline container copy (Cassandra & SQL)” preview from the ‘Preview Features’ list in the Azure portal.

You can learn more about intra-account container copy as well as refer how to create and monitor container copy jobs for your Azure Cosmos DB accounts.