{"id":20943,"date":"2018-10-16T05:00:32","date_gmt":"2018-10-16T05:00:32","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/vcblog\/?p=20943"},"modified":"2019-04-11T06:26:41","modified_gmt":"2019-04-11T06:26:41","slug":"standard-library-algorithms-changes-and-additions-in-c17","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cppblog\/standard-library-algorithms-changes-and-additions-in-c17\/","title":{"rendered":"Standard Library Algorithms: Changes and Additions in C++17"},"content":{"rendered":"<h4><em>Today we have a guest post from <a href=\"http:\/\/www.nuonsoft.com\/blog\/\">Marc Gregoire<\/a>, Software Architect at Nikon Metrology and Microsoft MVP since 2007.<\/em><\/h4>\n<p>&nbsp;<\/p>\n<p>The C++14 standard already contains a wealth of different kinds of algorithms. C++17 adds a couple more algorithms and updates some existing ones. This article explains what\u2019s new and what has changed in the C++17 Standard Library.<\/p>\n<h2>New Algorithms<\/h2>\n<h3>Sampling<\/h3>\n<p>C++17 includes the following new sampling algorithm:<\/p>\n<ul>\n<li><code>sample(first, last, out, n, gen)<\/code><\/li>\n<\/ul>\n<p>It uses the given random number generator (<code>gen<\/code>) to pick <code>n<\/code> random elements from a given range [<code>first<\/code>, <code>last<\/code>) and writes them to the given output iterator (<code>out<\/code>).<\/p>\n<p>Here is a simple piece of code that constructs a vector containing the integers 1 to 20. It then sets up a random number generator, and finally generates 10 sequences of 5 values, in which each value is randomly sampled from the <code>data<\/code> vector:<\/p>\n<pre>using namespace std;\r\n\r\nvector&lt;int&gt; data(20);\r\niota(begin(data), end(data), 1);\r\ncopy(cbegin(data), cend(data), ostream_iterator&lt;int&gt;(cout, \" \"));\r\ncout &lt;&lt; '\\n';\r\n\r\nrandom_device seeder;\r\nconst auto seed = seeder.entropy() ? seeder() : time(nullptr);\r\ndefault_random_engine generator(\r\n       static_cast&lt;default_random_engine::result_type&gt;(seed));\r\n\r\nconst size_t numberOfSamples = 5;\r\nvector&lt;int&gt; sampledData(numberOfSamples);\r\n\r\nfor (size_t i = 0; i &lt; 10; ++i)\r\n{\r\n    sample(cbegin(data), cend(data), begin(sampledData),\r\n           numberOfSamples, generator);\r\n    copy(cbegin(sampledData), cend(sampledData),\r\n         ostream_iterator&lt;int&gt;(cout, \" \"));\r\n    cout &lt;&lt; '\\n';\r\n}\r\n\r\nHere is an example of a possible output:<\/pre>\n<pre>1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20\r\n4 8 9 17 19\r\n4 7 12 13 18\r\n5 7 8 14 18\r\n1 4 5 10 20\r\n2 4 8 13 17\r\n2 3 4 5 20\r\n4 7 8 9 13\r\n1 7 8 10 15\r\n4 5 8 12 13\r\n1 3 8 10 19<\/pre>\n<h3>Iterating<\/h3>\n<p>The C++ Standard Library already included <code>for_each()<\/code> to process each element in a given range. C++17 adds a <code>for_each_n(first, n, func)<\/code> algorithm. It calls the given function object (<code>func<\/code>) for each element in the range given by a first iterator (<code>first<\/code>) and a number of elements (<code>n<\/code>). As such, it is very similar to <code>for_each()<\/code>, but <code>for_each_n()<\/code> only processes the first <code>n<\/code> elements of the range.<\/p>\n<p>Here is a simple example that generates a vector of 20 values, then uses <code>for_each_n()<\/code> to print the first 5 values to the console:<\/p>\n<pre>using namespace std;\r\n\r\nvector&lt;int&gt; data(20);\r\niota(begin(data), end(data), 1);\r\n\r\nfor_each_n(begin(data), 5,\r\n           [](const auto&amp; value) { cout &lt;&lt; value &lt;&lt; '\\n'; });<\/pre>\n<h3>Searching<\/h3>\n<p>C++17 includes a couple of specialized searchers, all defined in <code>&lt;functional&gt;<\/code>:<\/p>\n<ul>\n<li><code>default_searcher<\/code><\/li>\n<li><code>boyer_moore_searcher<\/code><\/li>\n<li><code>boyer_moore_horspool_searcher<\/code><\/li>\n<\/ul>\n<p>The Boyer-Moore searchers are often used to find a piece of text in a large block of text, and are usually more efficient than the default searcher. In practice, the two Boyer-Moore searchers are able to skip certain characters instead of having to compare each individual character. This gives these algorithms a sublinear complexity, making them much faster than the default searcher. See the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Boyer%E2%80%93Moore_string-search_algorithm\">Wikipedia article<\/a> for more details of the algorithm.<\/p>\n<p>To use these specialized searchers, you create an instance of one of them and pass that instance as the last parameter to <code>std::search()<\/code>, for example:<\/p>\n<pre>using namespace std;\r\n\r\nconst string haystack = \"Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.\";\r\nconst string needle = \"consectetur\";\r\n\r\nconst auto result = search(cbegin(haystack), cend(haystack),\r\n       boyer_moore_searcher(cbegin(needle), cend(needle)));\r\n\r\nif (result != cend(haystack))\r\n    cout &lt;&lt; \"Found it.\\n\";\r\nelse\r\n    cout &lt;&lt; \"Not found.\\n\";<\/pre>\n<p>If you want to carry out multiple searches on the same range, you can construct a single instance of <code>std::boyer_moore_searcher<\/code> and reuse it, rather than creating a new one for each <code>std::search()<\/code> call.<\/p>\n<h3>Generalized Sum Algorithms<\/h3>\n<h4>Scan<\/h4>\n<p>The following generalized sum algorithms have been added to the C++17 Standard Library:<\/p>\n<ul>\n<li><code>exclusive_scan(first, last, out, init[, bin_op])<\/code><\/li>\n<li><code>inclusive_scan(first, last, out[, bin_op[, init]])<\/code><\/li>\n<li><code>transform_exclusive_scan(first, last, out, init, bin_op, un_op)<\/code><\/li>\n<li><code>transform_inclusive_scan(first, last, out, bin_op, un_op[, init])<\/code><\/li>\n<\/ul>\n<p>Where <code>bin_op<\/code> is a binary operator (<code>std::plus&lt;&gt;()<\/code> by default), and un_op is a unary operator.<\/p>\n<p>All these algorithms calculate a sequence of sums of the elements in a given range [<code>first<\/code>, <code>last<\/code>), denoted as [e<sub>0<\/sub>, e<sub>n<\/sub>). The calculated sums are written to [<code>out<\/code>, <code>out + (last-first)<\/code>), denoted as [s<sub>0<\/sub>, s<sub>n<\/sub>). Suppose further that we denote the binary operator (<code>bin_op<\/code>) as \u2295. The <code>exclusive_scan()<\/code> algorithm then calculates the following sequence of sums:<\/p>\n<pre class=\"lang:default decode:true\">s0 = init \r\ns1 = init \u2295 e0\r\ns2 = init \u2295 e0 \u2295 e1 \r\n...\r\nsn-1 = init \u2295 e0 \u2295 e1 \u2295 ... \u2295 en\u22122<\/pre>\n<p>While <code>inclusive_scan()<\/code> calculates the following sums:<\/p>\n<pre class=\"lang:default decode:true\">s0 = init \u2295 e0 \r\ns1 = init \u2295 e0 \u2295 e1 \r\n... \r\nsn-1 = init \u2295 e0 \u2295 e1 \u2295 ... \u2295 en\u22121<\/pre>\n<p>The only difference is that <code>inclusive_scan()<\/code> includes the i<sup>th<\/sup> element in the i<sup>th<\/sup> sum, while <code>exclusive_scan()<\/code> does not include the i<sup>th<\/sup> element in the i<sup>th<\/sup> sum.<\/p>\n<p><code>exclusive_scan()<\/code> and <code>inclusive_scan()<\/code> are similar to <code>partial_sum()<\/code>. However, <code>partial_sum()<\/code> evaluates everything from left to right, while <code>exclusive_scan()<\/code> and <code>inclusive_scan()<\/code> evaluate everything in a non-deterministic order. That means that the result of these will be non-deterministic if the binary operator that is used is not associative. Because of the non-deterministic order, these algorithms can be executed in parallel by specifying a parallel execution policy, see later.<\/p>\n<p>The sums calculated by <code>inclusive_scan()<\/code> with <code>init<\/code> equal to 0 and an associative binary operator are exactly the same as the sums calculated by <code>partial_sum()<\/code>.<\/p>\n<p><code>transform_exclusive_scan()<\/code> and <code>transform_inclusive_scan()<\/code> are very similar. The only difference is that they apply the given unary operator before calculating the sum. Suppose the unary operator is denoted as a function call <em>f<\/em>(). The <code>transform_exclusive_scan()<\/code> algorithm then calculates the following sums sequence:<\/p>\n<pre class=\"lang:default decode:true \">s0 = init\r\ns1 = init \u2295 f(e0)\r\ns2 = init \u2295 f(e0) \u2295 f(e1)\r\n...\r\nsn-1 = init \u2295 f(e0) \u2295 f(e1) \u2295 ... \u2295 f(en\u22122)<\/pre>\n<p>&nbsp;<\/p>\n<p>And the transform_inclusive_scan() algorithm calculates the following sums:<\/p>\n<pre class=\"lang:default decode:true \">s0 = init \u2295 f(e0)\r\ns1 = init \u2295 f(e0) \u2295 f(e1)\r\n...\r\nsn-1 = init \u2295 f(e0) \u2295 f(e1) \u2295 ... \u2295 f(en\u22121)<\/pre>\n<p>&nbsp;<\/p>\n<h4>Reduce<\/h4>\n<p>Additionally, the following two reduce algorithms have been added:<\/p>\n<ul>\n<li><code> reduce(first, last[, init[, bin_op]])<\/code><\/li>\n<li><code> transform_reduce(first, last, init, bin_op, un_op)<\/code><\/li>\n<\/ul>\n<p>Where <code>bin_op<\/code> is a binary operator, <code>std::plus&lt;&gt;()<\/code> by default. These algorithms result in a single value, similar to <code>accumulate()<\/code>.<\/p>\n<p>Suppose again that the range [<code>first<\/code>, <code>last<\/code>) is denoted as [e<sub>0<\/sub>, e<sub>n<\/sub>). <code>reduce()<\/code> then calculates the following sum:<\/p>\n<pre class=\"lang:default decode:true\">init \u2295 e0 \u2295 e1 \u2295 ... \u2295 en\u22121<\/pre>\n<p>While transform_reduce() results in the following sum, assuming the unary operator is denoted as a function call <em>f<\/em>():<\/p>\n<pre class=\"lang:default decode:true\">init \u2295 f(e0) \u2295 f(e1) \u2295 ... \u2295 f(en\u22121)<\/pre>\n<p>Unlike <code>accumulate()<\/code>, <code>reduce()<\/code> supports parallel execution. The <code>accumulate()<\/code> algorithm always evaluates everything deterministically from left to right, while the evaluation order is non-deterministic for <code>reduce()<\/code>. A consequence is that the result of <code>reduce()<\/code> will be non-deterministic in case the binary operator is not associative or not commutative.<\/p>\n<p>The sum calculated by <code>reduce()<\/code> with <code>init<\/code> equal to 0 is exactly the same as the result of calling <code>accumulate()<\/code> as long as the binary operator that is used is associative and commutative.<\/p>\n<p>Finally, there is another set of overloads for <code>transform_reduce()<\/code>:<\/p>\n<ul>\n<li><code> transform_reduce(first1, last1, first2, init[, bin_op1, bin_op2])<\/code><\/li>\n<\/ul>\n<p>It requires two ranges: a range [<code>first1<\/code>, <code>last1<\/code>), denoted as [a<sub>0<\/sub>, a<sub>n<\/sub>), and a range starting at <code>first2<\/code>, denoted as [b<sub>0<\/sub>, b<sub>n<\/sub>). Suppose <code>bin_op1<\/code> (<code>std::plus&lt;&gt;()<\/code> by default) is denoted as \u2295, and <code>bin_op2<\/code> (<code>std::multiplies&lt;&gt;()<\/code> by default) is denoted as \u2296, then it calculates the following sum:<\/p>\n<pre class=\"lang:default decode:true\">init \u2295 (a0 \u2296 b0) \u2295 (a1 \u2296 b1) \u2295 ... \u2295 (an-1 \u2296 bn-1)<\/pre>\n<h2>Parallel Algorithms<\/h2>\n<p>A major addition to the C++17 Standard Library is support for parallel execution of more than 60 of its algorithms, such as <code>sort()<\/code>, <code>all_of()<\/code>, <code>find()<\/code>, <code>transform()<\/code>, \u2026<\/p>\n<p>If a Standard Library algorithm supports parallel execution, then it accepts an execution policy as its first parameter. This policy determines to what degree the algorithm may parallelize or vectorize its execution. Currently, the following policy types and instances are defined in the <code>std::execution<\/code> namespace in the <code>&lt;execution&gt;<\/code> header:<\/p>\n<table>\n<tbody>\n<tr>\n<td width=\"201\"><strong><em>Execution Policy Type<\/em><\/strong><\/td>\n<td width=\"123\"><strong>Global Instance<\/strong><\/td>\n<td width=\"280\"><strong>Description<\/strong><\/td>\n<\/tr>\n<tr>\n<td width=\"201\"><em>sequenced_policy<\/em><\/td>\n<td width=\"123\"><code>seq<\/code><\/td>\n<td width=\"280\">No parallel execution is allowed.<\/td>\n<\/tr>\n<tr>\n<td width=\"201\"><em>parallel_policy<\/em><\/td>\n<td width=\"123\"><code>par<\/code><\/td>\n<td width=\"280\">Parallel execution is allowed.<\/td>\n<\/tr>\n<tr>\n<td width=\"201\"><em>parallel_unsequenced_policy<\/em><\/td>\n<td width=\"123\"><code>par_unseq<\/code><\/td>\n<td width=\"280\">Parallel and vectorized execution is allowed. Execution is also allowed to switch between different threads.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The <code>parallel_unsequenced_policy<\/code> imposes a lot of restrictions on what the algorithm\u2019s function callbacks are allowed to do. With that policy, the calls to the callbacks are unsequenced. As such, its callbacks are not allowed to perform memory allocation\/deallocation, acquire mutexes, and more. The other policies do not have such restrictions, and in fact guarantee that their callback calls are sequenced, although of course they can be in a non-deterministic order. In any case, you are responsible to prevent data races and deadlocks.<\/p>\n<p>Using these parallel policies is straightforward. Here is a quick example that generates a <code>vector<\/code> of 1 billion double values, then uses the <code>std::transform()<\/code> algorithm to calculate the square root of each value in parallel:<\/p>\n<pre>using namespace std;\r\n\r\nvector&lt;double&gt; data(1'000'000'000);\r\niota(begin(data), end(data), 1);\r\n\r\ntransform(execution::par_unseq, begin(data), end(data), begin(data),\r\n          [](const auto&amp; value) { return sqrt(value); });<\/pre>\n<p>If you run this piece of code on an 8-core machine, the CPU load can look as follows. The peak you see on all eight cores is the parallel execution of the call to <code>std::transform()<\/code>.<\/p>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/9\/2019\/02\/graph.png\"><img decoding=\"async\" class=\"wp-image-21135 size-large aligncenter\" src=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/9\/2019\/02\/graph-1024x460.png\" alt=\"\" width=\"879\" height=\"395\" \/><\/a><\/p>\n<h2>Utility Functions<\/h2>\n<p>C++17 also includes a couple of handy utility functions that are not really algorithms, but still useful to know.<\/p>\n<h3>clamp()<\/h3>\n<p><code>std::clamp(value, low, high)<\/code> is defined in the <code>&lt;algorithm&gt;<\/code> header. It ensures that a given value is within a given range [<code>low<\/code>, <code>high<\/code>]. The result of calling <code>clamp()<\/code> is:<\/p>\n<ul>\n<li>a reference to <code>low<\/code> if <code>value<\/code> &lt; <code>low<\/code><\/li>\n<li>a reference to <code>high<\/code> if <code>value<\/code> &gt; <code>high<\/code><\/li>\n<li>otherwise, a reference to the given <code>value<\/code><\/li>\n<\/ul>\n<p>One use-case is to clamp audio samples to a 16-bit range:<\/p>\n<pre>using namespace std;\r\nconst int low = -32'768;\r\nconst int high = 32'767;\r\ncout &lt;&lt; clamp(12'000, low, high) &lt;&lt; '\\n';\r\ncout &lt;&lt; clamp(-36'000, low, high) &lt;&lt; '\\n';\r\ncout &lt;&lt; clamp(40'000, low, high) &lt;&lt; '\\n';<\/pre>\n<p>The output of this code snippet is as follows:<\/p>\n<pre>12000\r\n-32768\r\n32767<\/pre>\n<h3>gcd() and lcm()<\/h3>\n<p><code>std::gcd()<\/code> returns the greatest common divisor of two integer types, while <code>lcm()<\/code> returns the least common multiple of two integer types. Both algorithms are defined in the <code>&lt;numeric&gt;<\/code> header.<\/p>\n<p>Using these algorithms is straightforward, for example:<\/p>\n<pre>cout &lt;&lt; gcd(24, 44) &lt;&lt; '\\n';\r\ncout &lt;&lt; lcm(24, 44) &lt;&lt; '\\n';<\/pre>\n<p>The output is as follows:<\/p>\n<pre>4\r\n264<\/pre>\n<h2>Removed Algorithms<\/h2>\n<p>C++17 has removed one algorithm: std::random_shuffle(). This algorithm was previously already marked as deprecated by C++14. You should use <code>std::shuffle()<\/code> instead.<\/p>\n<h2>Further Reading Material<\/h2>\n<p>Have a look at my book, \u201c<a href=\"https:\/\/amzn.to\/2JjvkWe\">Professional C++, 4<sup>th<\/sup> Edition<\/a>\u201d, published by Wiley\/Wrox, for a more in-depth overview of all the functionality provided by the C++17 Standard Library. It also includes a description of all language features that have been added by C++17.<\/p>\n<p>Additionally, you can also read more about certain C++17 features on my <a href=\"http:\/\/www.nuonsoft.com\/blog\/tag\/c17\/\">C++17 blog post series<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Today we have a guest post from Marc Gregoire, Software Architect at Nikon Metrology and Microsoft MVP since 2007. &nbsp; The C++14 standard already contains a wealth of different kinds of algorithms. C++17 adds a couple more algorithms and updates some existing ones. This article explains what\u2019s new and what has changed in the C++17 [&hellip;]<\/p>\n","protected":false},"author":289,"featured_media":35994,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[512],"tags":[],"class_list":["post-20943","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-general-cpp-series"],"acf":[],"blog_post_summary":"<p>Today we have a guest post from Marc Gregoire, Software Architect at Nikon Metrology and Microsoft MVP since 2007. &nbsp; The C++14 standard already contains a wealth of different kinds of algorithms. C++17 adds a couple more algorithms and updates some existing ones. This article explains what\u2019s new and what has changed in the C++17 [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts\/20943","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/users\/289"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/comments?post=20943"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts\/20943\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/media\/35994"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/media?parent=20943"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/categories?post=20943"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/tags?post=20943"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}