{"id":55975,"date":"2009-10-12T09:03:38","date_gmt":"2009-10-12T09:03:38","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/pfxteam\/2009\/10\/12\/parallelized-map-and-filter-operations\/"},"modified":"2009-10-12T09:03:38","modified_gmt":"2009-10-12T09:03:38","slug":"parallelized-map-and-filter-operations","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/dotnet\/parallelized-map-and-filter-operations\/","title":{"rendered":"Parallelized Map and Filter Operations"},"content":{"rendered":"<p>Common operations like map and filter are available in parallelized form through PLINQ, though the names differ.&nbsp; A map can be achieved with PLINQ&rsquo;s Select operator, and a filter with PLINQ&rsquo;s Where operator.<\/p>\n<p>For example, I could implement a ParallelMap operation that takes in one array and returns another as follows:<\/p>\n<blockquote>\n<p>public static TOutput [] ParallelMap&lt;TInput,TOutput&gt;(     <br>&nbsp;&nbsp;&nbsp; TInput [] input, Func&lt;TInput,TOutput&gt; map)      <br>{      <br>&nbsp;&nbsp;&nbsp; return input.AsParallel().Select(map).ToArray();      <br>}<\/p>\n<\/blockquote>\n<p>and a ParallelFilter operation could be implemented in a similar fashion, based on PLINQ&rsquo;s Where operator:<\/p>\n<blockquote>\n<p>public static TInput [] ParallelFilter&lt;TInput&gt;(     <br>&nbsp;&nbsp;&nbsp; TInput [] input, Func&lt;TInput,Boolean&gt; filter)      <br>{      <br>&nbsp;&nbsp;&nbsp; return input.AsParallel().Where(filter).ToArray();      <br>}<\/p>\n<\/blockquote>\n<p>In general, PLINQ is the recommended way to achieve such parallelization.&nbsp; Not only is it useful for individual operations like this, but it&rsquo;s also a great way to compose operations.&nbsp; For example, if I wanted to write a ParallelFilterAndMap, I could achieve that as follows:<\/p>\n<blockquote>\n<p>public static TOutput [] ParallelFilterAndMap&lt;TInput,TOutput&gt;(     <br>&nbsp;&nbsp;&nbsp; TInput [] input, Func&lt;TInput,Boolean&gt; filter, Func&lt;TInput,TOutput&gt; map)      <br>{      <br>&nbsp;&nbsp;&nbsp; return input.AsParallel().Where(filter).Select(map).ToArray();      <br>}<\/p>\n<\/blockquote>\n<p>It is also possible to build such operations on top of Parallel.*.&nbsp; For example, here&rsquo;s ParallelMap re-implemented in terms of Parallel.For:<\/p>\n<blockquote>\n<p>public static TOutput [] ParallelMap&lt;TInput,TOutput&gt;(     <br>&nbsp;&nbsp;&nbsp; TInput [] input, Func&lt;TInput,TOutput&gt; map)      <br>{      <br>&nbsp;&nbsp;&nbsp; var output = new TOutput[input.Length];      <br>&nbsp;&nbsp;&nbsp; Parallel.For(0, input.Length, i =&gt; output[i] = map(input[i]));      <br>&nbsp;&nbsp;&nbsp; return output;      <br>}<\/p>\n<\/blockquote>\n<p>and ParallelFilterAndMap re-implemented in terms of Parallel.ForEach and ConcurrentBag&lt;T&gt;:<\/p>\n<blockquote>\n<p>public static TInput [] ParallelFilterAndMap&lt;TInput,TOutput&gt;(     <br>&nbsp;&nbsp;&nbsp; TInput [] input, Func&lt;TInput,Boolean&gt; filter, Func&lt;TInput,TOutput&gt; map)      <br>{      <br>&nbsp;&nbsp;&nbsp; var output = new ConcurrentBag&lt;T&gt;();      <br>&nbsp;&nbsp;&nbsp; Parallel.ForEach(input, item =&gt;      <br>&nbsp;&nbsp;&nbsp; {      <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; if (filter(item)) output.Add(map(item));      <br>&nbsp;&nbsp;&nbsp; });      <br>&nbsp;&nbsp;&nbsp; return output.ToArray();      <br>}<\/p>\n<\/blockquote>\n<p>As mentioned, in most cases you&rsquo;re better off using PLINQ for these kinds of operations.&nbsp; However, there may be cases where a Parallel.*-based version is best.&nbsp; For example, let&rsquo;s say that you wanted to implement an in-place parallel map operation, e.g. a map operation that maps from TData to TData, and that stores the new output into the input array.&nbsp; PLINQ doesn&rsquo;t modify the input data source, so if you wanted that functionality, you could build it on top of Parallel.For:<\/p>\n<blockquote>\n<p>public static void ParallelMap&lt;TData&gt;(     <br>&nbsp;&nbsp;&nbsp; TData [] data, Func&lt;TData,TData&gt; map)      <br>{      <br>&nbsp;&nbsp;&nbsp; Parallel.For(0, data.Length, i =&gt; data[i] = map(data[i]));      <br>}<\/p>\n<\/blockquote>\n<p>There are a few other subtle differences that you could take advantage of by building on Parallel.For\/ForEach instead of on PLINQ.&nbsp; For example, PLINQ uses a static number of threads for the processing of a given query (which defaults to Environment.ProcessorCount), whereas the number of threads used by Parallel.* may vary over time as influenced by the ThreadPool&rsquo;s thread injection and retirement logic.&nbsp; Parallel.* can also be targeted to a specific and custom TaskScheduler, something not supported by PLINQ in .NET 4 (PLINQ always runs on TaskScheduler.Default, which is based on the .NET 4 ThreadPool).<\/p>\n<p>Happy coding.<\/p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Common operations like map and filter are available in parallelized form through PLINQ, though the names differ.&nbsp; A map can be achieved with PLINQ&rsquo;s Select operator, and a filter with PLINQ&rsquo;s Where operator. For example, I could implement a ParallelMap operation that takes in one array and returns another as follows: public static TOutput [] [&hellip;]<\/p>\n","protected":false},"author":360,"featured_media":58792,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[7908],"tags":[7907,7911,7909,7910,7912],"class_list":["post-55975","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-pfxteam","tag-net-4","tag-code-samples","tag-parallel-extensions","tag-plinq","tag-task-parallel-library"],"acf":[],"blog_post_summary":"<p>Common operations like map and filter are available in parallelized form through PLINQ, though the names differ.&nbsp; A map can be achieved with PLINQ&rsquo;s Select operator, and a filter with PLINQ&rsquo;s Where operator. For example, I could implement a ParallelMap operation that takes in one array and returns another as follows: public static TOutput [] [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/55975","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/360"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=55975"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/55975\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/58792"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=55975"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=55975"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=55975"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}