{"id":152,"date":"2019-11-11T09:00:03","date_gmt":"2019-11-11T17:00:03","guid":{"rendered":"http:\/\/devblogs.microsoft.com\/cosmosdb\/?p=152"},"modified":"2021-02-19T14:02:55","modified_gmt":"2021-02-19T22:02:55","slug":"introducing-bulk-support-in-the-net-sdk","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cosmosdb\/introducing-bulk-support-in-the-net-sdk\/","title":{"rendered":"Introducing Bulk support in the .NET SDK"},"content":{"rendered":"

The Azure Cosmos DB .NET SDK has recently released<\/a> Bulk support in version 3.4.0.<\/p>\n

What exactly is \u201cBulk\u201d?<\/h2>\n

Bulk refers to scenarios that require a high degree of throughput, where you need to dump a big volume of data<\/strong>, and you need to do it with as much throughput as possible<\/strong>.<\/p>\n

Are you doing a nightly dump of 2 million records into your Cosmos DB container? That is bulk.<\/p>\n

Are you processing a stream of data that comes in batches of 100 thousand items you need to update? That is bulk too.<\/p>\n

Are you dynamically generating groups of operations that execute concurrently? Yeah, that is also bulk.<\/p>\n

What is NOT \u201cBulk\u201d?<\/h2>\n

Bulk is about throughput, not latency<\/strong>. So, if you are working on a latency-sensitive scenario, where point operations need to resolve as quick as possible, then bulk is not the right tool.<\/p>\n

How do I enable Bulk in .NET SDK V3?<\/h2>\n

Enabling bulk is rather easy, when you create the CosmosClient<\/strong>, toggle the AllowBulkExecution<\/strong> flag in the CosmosClientOptions<\/strong> like so:<\/p>\n

CosmosClientOptions options = new CosmosClientOptions() { AllowBulkExecution = true };\r\nCosmosClient cosmosClient = new CosmosClient(connectionString, options);<\/pre>\n
After that, all you need to do is get an instance of a Container<\/strong> where you want to perform the operations, and create a list of Tasks<\/strong> that represent the operations you want to perform based on your input data (I\u2019m removing obtaining the information from the source for simplicity\u2019s sake and because the origin of the data will vary in each case):<\/p>\n
Container container = cosmosClient.GetContainer(\"myDb\", \"myCollection\");\r\n\r\n\/\/ Assuming your have your data available to be inserted or read\r\nList<Task> concurrentTasks = new List<Task>();\r\nforeach(Item itemToInsert in ReadYourData())\r\n{\r\n    concurrentTasks.Add(container.CreateItemAsync(itemToInsert, new PartitionKey(itemToInsert.MyPk)));\r\n}\r\n\r\nawait Task.WhenAll(concurrentTasks);\r\n<\/pre>\n
Wait, what is happening with those Tasks?<\/h2>\n
Normally, if you issue a hundred CreateItemAsync operations in parallel<\/strong>, each one will generate a service request and response independently. The illustration below shows what execution looks like when individual threads individually insert new items.<\/p>\n
\"Operations<\/p>\n
But the magic here happens when the SDK detects that the Bulk mode is enabled.<\/p>\n
The SDK will create batches, all the concurrent<\/strong> operations will be grouped<\/strong> by physical partition<\/a> affinity and distributed across these batches. When a batch fills up, it gets dispatched, and a new batch is created to be filled with more concurrent operations. Each batch will contain many operations, so this greatly reduces the amount of back end requests<\/strong>. There could be many batches being dispatched in parallel targeting different partitions, so the more evenly distributed the operations, the better results.<\/p>\n
Now, here comes the best part.<\/p>\n
When you (the user) call an operation method (CreateItemAsync in this example), the SDK returns a Task<\/strong>. In a normal, non-bulk CosmosClient, this Task represents the service request for that operation, and completes when the request get the response. But in Bulk, that Task holds the promise of a result<\/strong>, it does not map to a service request. The SDK is grouping and squeezing operations into batches.<\/p>\n
When a batch is completed, the SDK then unwraps<\/strong> all the results for all the operations the batch contained and completes the related Tasks<\/strong> with the result. In the illustration below, you can see how batch makes things more efficient and allows you to consume more throughput than you could if done as individual threads.<\/p>\n
\"Operations<\/p>\n
From the user perspective, its transparent.<\/p>\n
What are the caveats?<\/h2>\n
Since Bulk is geared towards obtaining as much throughput as possible, and the volume of data will be higher than executing the operations individually, the provisioned throughput<\/strong> (RU\/s<\/a>) consumed will be higher. Make sure you adjust it based on the volume of operations you want to push.<\/p>\n
If your container gets throttled during a Bulk process, it means that now the bottleneck of you getting a higher throughput and pushing even more operations is on the provisioned throughput, raising it will let you push even more operations per second.\u00a0The .NET SDK will automatically retry when throttling happens, but the overall effect will be that processing the data will be slower.<\/p>\n
Another caveat is the size of the documents<\/strong>. The batches that the SDK creates to optimize throughput have a current maximum of 2Mb or 100 operations per batch, the smaller the documents, the greater the optimization that can be achieved (the bigger the documents, the more batches need to be used).<\/p>\n
And finally, the amount of operations<\/strong>. As it was mentioned before, the SDK will construct batches and group operations, when the batch is full, it will get dispatched, but if the batch doesn’t fill up, there is a timer<\/strong> that will dispatch it to make sure they complete. This timer currently is 100 milliseconds. So if the batch does not get filled up (for example, you are just sending 50 concurrent operations), then the overall latency might be affected.<\/p>\n
In general, to troubleshoot<\/strong> this scenario it is good to:<\/p>\n