We’re excited to announce the release of the modern Azure Storage Data Movement library, designed to simplify your data transfer experience when using Azure Blob Storage and Azure Files. This new library and the extension libraries for Azure Storage and Azure Files are now available as of February 11, 2025. In this blog post, we’ll explore the benefits and how to get started with the library.
Why use the new and improved Data Movement library?
Convenience with high performance
The modern Azure Storage Data Movement library is the perfect choice for uploading, downloading, and copying of blobs and files to, from, and between storage accounts. For your convenience, methods and objects are included in the client library to perform the following operations:
- Setting the number of parallel operations for each transfer
- Tracking transfer progress
- Pause/resume functionality
- Checkpointing
Compared to the standard Azure Storage client libraries, the Data Movement library targets specific advanced data transfer scenarios: directory level movement, resource consumption control, and large-scale data movement. Unlike AzCopy, PowerShell or other executable tools, the Data Movement library provides a .NET package and extension libraries that can be easily integrated into customers’ on-premises or cloud service.
Modernization
The new Azure Storage Data Movement libraries share infrastructure with modern, v12 Azure Storage libraries, simplifying the process of transferring and copying data to and from Azure Storage. In addition, the libraries integrate with the modern Azure Identity libraries, helping you stay up-to-date with modern security practices.
With the legacy version currently on track to be retired in March 2026, the modern version of the Data Movement library supports new Azure Storage features and services going forward.
Get started with the new Azure Storage Data Movement library
The collection of modern Azure Storage Data Movement libraries includes a common client library and specialized extension libraries for Azure Blob Storage and Azure Files. The common library handles core data transfer functions, while the extension libraries offer extra features specific to Azure Blob Storage and Azure Files.
The upcoming section covers samples for getting started with the Data Movement library. Microsoft Entra token-based authentication is used in each of the samples, with the help of the Azure Identity library. For the samples, the role of Azure Storage Data Contributor (or an elevated role) is needed to perform the operations.
Installation
Use the following commands to install the required packages:
dotnet add package Azure.Storage.DataMovement.Blobs
dotnet add package Azure.Identity
Add using
directives
Add these directives to the top of your code file:
using Azure;
using Azure.Core;
using Azure.Identity;
using Azure.Storage.DataMovement;
using Azure.Storage.DataMovement.Blobs;
The TransferManager
object
The TransferManager
serves as the primary class for initiating and managing various types of transfers, such as uploads, downloads, and copies. Let’s take a look at creating a TransferManager
object to interact with a local file system, Azure Blob Storage, or Azure Files:
TransferManager transferManager = new(new TransferManagerOptions());
Create a StorageResource
object for Azure Blob Storage
StorageResource
is the foundational class for all storage resources, including blobs and files. For Azure Blob Storage, the BlobsStorageResourceProvider
creates StorageResource
instances for a blob container, block blob, append blob, or page blob.
// Create a token credential
DefaultAzureCredential tokenCredential = new();
BlobsStorageResourceProvider blobsProvider = new(tokenCredential);
// Get a container resource
StorageResource container = await blobsProvider.FromContainerAsync(
new Uri("http://<storage-account-name>.blob.core.windows.net/sample-container"));
// Get a block blob resource - default is block blob
StorageResource blockBlob = await blobsProvider.FromBlobAsync(
new Uri("http://<storage-account-name>.blob.core.windows.net/sample-container/sample-block-blob"),
new BlockBlobStorageResourceOptions());
Example: Upload a local file to a blob
To initiate a new transfer, use the TransferManager.StartTransferAsync
method. This method returns a TransferOperation
object representing the transfer. The TransferOperation
object allows you to monitor the transfer progress and obtain the transfer ID. The transfer ID is a unique identifier for resuming or pausing a transfer.
Transfers involve a source and a destination, both of which are StorageResource
types. The source and destination can be either StorageResourceContainer
or StorageResourceItem
, and they must be the same. For instance, if the source is a blob container, the destination must also be a blob container.
The following example shows how to start a new transfer to upload a local file to a blob:
DefaultAzureCredential tokenCredential = new();
TransferManager transferManager = new(new TransferManagerOptions());
BlobsStorageResourceProvider blobsProvider = new(tokenCredential);
string localFilePath = "C:/path/to/file.txt";
string blobUri = "https://<storage-account-name>.blob.core.windows.net/sample-container/sample-blob";
TransferOperation transferOperation = await transferManager.StartTransferAsync(
sourceResource: LocalFilesStorageResourceProvider.FromFile(localFilePath),
destinationResource: await blobsProvider.FromBlobAsync(new Uri(blobUri)));
await transferOperation.WaitForCompletionAsync();
Conclusion
The new Azure Storage Data Movement library provides .NET developers with an easy, powerful way to facilitate data transfers in your applications using Azure Blob Storage and Azure Files.
We encourage all users of the legacy version of the Azure Storage Data Movement library to take advantage of the significant improvements in the update and migrate to the new version.
Resources
- Article: Transfer data with the Data Movement library
- The Azure Storage Data Movement Common library on GitHub
- Azure.Storage.DataMovement.Blobs Documentation
- Azure.Storage.DataMovement.Files.Shares documentation
- Migration guide from v1 to v2 for Data Movement Library
For feature requests, bug reports, or general support, open an issue in the repository on GitHub.
0 comments
Be the first to start the discussion.