February 12th, 2025

Simplify your .NET data transfers with the new Azure Storage Data Movement library

Charles Barnett
Program Manager 2

We’re excited to announce the release of the modern Azure Storage Data Movement library, designed to simplify your data transfer experience when using Azure Blob Storage and Azure Files. This new library and the extension libraries for Azure Storage and Azure Files are now available as of February 11, 2025. In this blog post, we’ll explore the benefits and how to get started with the library.

Why use the new and improved Data Movement library?

Convenience with high performance

The modern Azure Storage Data Movement library is the perfect choice for uploading, downloading, and copying of blobs and files to, from, and between storage accounts. For your convenience, methods and objects are included in the client library to perform the following operations:

  • Setting the number of parallel operations for each transfer
  • Tracking transfer progress
  • Pause/resume functionality
  • Checkpointing

Compared to the standard Azure Storage client libraries, the Data Movement library targets specific advanced data transfer scenarios: directory level movement, resource consumption control, and large-scale data movement. Unlike AzCopy, PowerShell or other executable tools, the Data Movement library provides a .NET package and extension libraries that can be easily integrated into customers’ on-premises or cloud service.

Modernization

The new Azure Storage Data Movement libraries share infrastructure with modern, v12 Azure Storage libraries, simplifying the process of transferring and copying data to and from Azure Storage. In addition, the libraries integrate with the modern Azure Identity libraries, helping you stay up-to-date with modern security practices.

With the legacy version currently on track to be retired in March 2026, the modern version of the Data Movement library supports new Azure Storage features and services going forward.

Get started with the new Azure Storage Data Movement library

The collection of modern Azure Storage Data Movement libraries includes a common client library and specialized extension libraries for Azure Blob Storage and Azure Files. The common library handles core data transfer functions, while the extension libraries offer extra features specific to Azure Blob Storage and Azure Files.

The upcoming section covers samples for getting started with the Data Movement library. Microsoft Entra token-based authentication is used in each of the samples, with the help of the Azure Identity library. For the samples, the role of Azure Storage Data Contributor (or an elevated role) is needed to perform the operations.

Installation

Use the following commands to install the required packages:

dotnet add package Azure.Storage.DataMovement.Blobs 
dotnet add package Azure.Identity 

Add using directives

Add these directives to the top of your code file:

using Azure;
using Azure.Core;
using Azure.Identity;
using Azure.Storage.DataMovement;
using Azure.Storage.DataMovement.Blobs;

The TransferManager object

The TransferManager serves as the primary class for initiating and managing various types of transfers, such as uploads, downloads, and copies. Let’s take a look at creating a TransferManager object to interact with a local file system, Azure Blob Storage, or Azure Files:

TransferManager transferManager = new(new TransferManagerOptions()); 

Create a StorageResource object for Azure Blob Storage

StorageResource is the foundational class for all storage resources, including blobs and files. For Azure Blob Storage, the BlobsStorageResourceProvider creates StorageResource instances for a blob container, block blob, append blob, or page blob.

// Create a token credential 
DefaultAzureCredential tokenCredential = new();

BlobsStorageResourceProvider blobsProvider = new(tokenCredential); 

// Get a container resource 
StorageResource container = await blobsProvider.FromContainerAsync( 
    new Uri("http://<storage-account-name>.blob.core.windows.net/sample-container")); 

// Get a block blob resource - default is block blob 
StorageResource blockBlob = await blobsProvider.FromBlobAsync( 
    new Uri("http://<storage-account-name>.blob.core.windows.net/sample-container/sample-block-blob"), 
    new BlockBlobStorageResourceOptions()); 

Example: Upload a local file to a blob

To initiate a new transfer, use the TransferManager.StartTransferAsync method. This method returns a TransferOperation object representing the transfer. The TransferOperation object allows you to monitor the transfer progress and obtain the transfer ID. The transfer ID is a unique identifier for resuming or pausing a transfer.

Transfers involve a source and a destination, both of which are StorageResource types. The source and destination can be either StorageResourceContainer or StorageResourceItem, and they must be the same. For instance, if the source is a blob container, the destination must also be a blob container.

The following example shows how to start a new transfer to upload a local file to a blob:

DefaultAzureCredential tokenCredential = new();

TransferManager transferManager = new(new TransferManagerOptions()); 
BlobsStorageResourceProvider blobsProvider = new(tokenCredential); 
string localFilePath = "C:/path/to/file.txt"; 
string blobUri = "https://<storage-account-name>.blob.core.windows.net/sample-container/sample-blob"; 

TransferOperation transferOperation = await transferManager.StartTransferAsync( 
    sourceResource: LocalFilesStorageResourceProvider.FromFile(localFilePath), 
    destinationResource: await blobsProvider.FromBlobAsync(new Uri(blobUri)));

await transferOperation.WaitForCompletionAsync();

Conclusion

The new Azure Storage Data Movement library provides .NET developers with an easy, powerful way to facilitate data transfers in your applications using Azure Blob Storage and Azure Files.

We encourage all users of the legacy version of the Azure Storage Data Movement library to take advantage of the significant improvements in the update and migrate to the new version.

Resources

For feature requests, bug reports, or general support, open an issue in the repository on GitHub.

Author

Charles Barnett
Program Manager 2

Charles has a passion for storage and the Azure cloud as a Product Manager 2 in Azure Storage.

0 comments