ParallelExtensionsExtras Tour – #14 – SingleItemPartitioner
(The full set of ParallelExtensionsExtras Tour posts is available here.)
In a previous ParallelExtensionsExtras Tour blog post, we talked about implementing a custom partitioner for BlockingCollection<T>. Custom partitioning is an advanced but important feature supported by both Parallel.ForEach and PLINQ, as it allows the developer full control over how data is distributed during parallel processing. .NET 4 includes several high-quality partitioning algorithms that are built-in and used by default, but sometimes you do want to tweak the employed behaviors, and that’s where the Partitioner<T> and OrderablePartitioner<T> types come in.
For example, the built-in partitioning algorithms do chunking in order to amortize the cost of synchronization across multiple data elements. For some scenarios, however, that chunking could lead to less-than-ideal behavior, and thus you might want to write a custom partitioner that doesn’t do any chunking, that instead always hands out a single data element at a time to whichever thread is ready next to process more data. ParallelExtensionsExtras includes just such a partitioner, SingleItemPartitioner, in the SingleItemPartitioner.cs file. Rather than explain this type in detail here, I’ll instead point you to a new article in Dr. Dobbs “Custom Parallel Partitioning in .NET 4” that walks through the principles of custom partitioning and SingleItemPartitioner’s implementation in detail.
Enjoy!
Light
Dark
0 comments