(The full set of ParallelExtensionsExtras Tour posts is available here.) 

In a previous ParallelExtensionsExtras Tour blog post, we talked about implementing a custom partitioner for BlockingCollection<T>.  Custom partitioning is an advanced but important feature supported by both Parallel.ForEach and PLINQ, as it allows the developer full control over how data is distributed during parallel processing.  .NET 4 includes several high-quality partitioning algorithms that are built-in and used by default, but sometimes you do want to tweak the employed behaviors, and that’s where the Partitioner<T> and OrderablePartitioner<T> types come in.

For example, the built-in partitioning algorithms do chunking in order to amortize the cost of synchronization across multiple data elements.  For some scenarios, however, that chunking could lead to less-than-ideal behavior, and thus you might want to write a custom partitioner that doesn’t do any chunking, that instead always hands out a single data element at a time to whichever thread is ready next to process more data.  ParallelExtensionsExtras includes just such a partitioner, SingleItemPartitioner, in the SingleItemPartitioner.cs file.  Rather than explain this type in detail here, I’ll instead point you to a new article in Dr. Dobbs “Custom Parallel Partitioning in .NET 4” that walks through the principles of custom partitioning and SingleItemPartitioner’s implementation in detail.

Enjoy!