ParallelExtensionsExtras Tour – #14 – SingleItemPartitioner

Avatar

Stephen

(The full set of ParallelExtensionsExtras Tour posts is available here.) 

In a previous ParallelExtensionsExtras Tour blog post, we talked about implementing a custom partitioner for BlockingCollection<T>.  Custom partitioning is an advanced but important feature supported by both Parallel.ForEach and PLINQ, as it allows the developer full control over how data is distributed during parallel processing.  .NET 4 includes several high-quality partitioning algorithms that are built-in and used by default, but sometimes you do want to tweak the employed behaviors, and that’s where the Partitioner<T> and OrderablePartitioner<T> types come in.

For example, the built-in partitioning algorithms do chunking in order to amortize the cost of synchronization across multiple data elements.  For some scenarios, however, that chunking could lead to less-than-ideal behavior, and thus you might want to write a custom partitioner that doesn’t do any chunking, that instead always hands out a single data element at a time to whichever thread is ready next to process more data.  ParallelExtensionsExtras includes just such a partitioner, SingleItemPartitioner, in the SingleItemPartitioner.cs file.  Rather than explain this type in detail here, I’ll instead point you to a new article in Dr. Dobbs “Custom Parallel Partitioning in .NET 4” that walks through the principles of custom partitioning and SingleItemPartitioner’s implementation in detail.

Enjoy!

Avatar
Stephen Toub

Software Engineer, .NET

Follow Stephen   

0 comments

    Leave a comment