{"id":1363,"date":"2014-03-31T07:00:00","date_gmt":"2014-03-31T07:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/2014\/03\/31\/a-puzzle-involving-dynamic-programming-or-maybe-it-doesnt\/"},"modified":"2014-03-31T07:00:00","modified_gmt":"2014-03-31T07:00:00","slug":"a-puzzle-involving-dynamic-programming-or-maybe-it-doesnt","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20140331-00\/?p=1363","title":{"rendered":"A puzzle involving dynamic programming, or maybe it doesn&#039;t"},"content":{"rendered":"<p>\nHere&#8217;s a programming challenge:\n<\/p>\n<blockquote CLASS=\"q\">\n<p>\nEvaluate the following recurrence relation efficiently\nfor a given array\n<span TITLE=\"x sub 0 to x sub (n-1)\">\n[<var>x<\/var><sub>0<\/sub>, &hellip;, <var>x<\/var><sub><var>n<\/var>&minus;1<\/sub>]\n<\/span>\nand integer\n<span TITLE=\"k\"><var>k<\/var><\/span>.\n<\/p>\n<table BORDER=\"0\" CELLSPACING=\"0\" CELLPADDING=\"0\" STYLE=\"line-height: 200%\">\n<tr TITLE=\"f([x sub 0, x sub 1], k) = (x sub 0 + x sub 1) \/ 2 for all k\">\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>0<\/sub>, <var>x<\/var><sub>1<\/sub>],&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>k<\/var>) =<sub>&nbsp;<\/sub>\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    (<var>x<\/var><sub>0<\/sub> + <var>x<\/var><sub>1<\/sub>) \/ 2&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    for all <var>k<\/var>, <var>n<\/var> = 2.<sub>&nbsp;<\/sub>\n    <\/td>\n<\/tr>\n<tr>= 3&#8243;&gt;<\/p>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>0<\/sub>, &hellip;,\n               <var>x<\/var><sub><var>n<\/var>&minus;1<\/sub>],&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>k<\/var>) =<sub>&nbsp;<\/sub>\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    (<var>x<\/var><sub>0<\/sub> + <var>x<\/var><sub><var>n<\/var>&minus;1<\/sub>) \/ 2 +&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>0<\/sub>, &hellip;, <var>x<\/var><sub><var>n<\/var>&minus;2<\/sub>], <var>k<\/var> + 1) \/ 2 +&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>1<\/sub>, &hellip;, <var>x<\/var><sub><var>n<\/var>&minus;1<\/sub>], <var>k<\/var> + 1) \/ 2&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    for even <var>k<\/var>, <var>n<\/var> &ge; 3.<sub>&nbsp;<\/sub>\n    <\/td>\n<\/tr>\n<tr>= 3&#8243;&gt;<\/p>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>0<\/sub>, &hellip;,\n               <var>x<\/var><sub><var>n<\/var>&minus;1<\/sub>],&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>k<\/var>) =<sub>&nbsp;<\/sub>\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    &nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>0<\/sub>, &hellip;, <var>x<\/var><sub><var>n<\/var>&minus;2<\/sub>], <var>k<\/var> + 1) \/ 2 +&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>1<\/sub>, &hellip;, <var>x<\/var><sub><var>n<\/var>&minus;1<\/sub>], <var>k<\/var> + 1) \/ 2&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    for odd <var>k<\/var>, <var>n<\/var> &ge; 3.<sub>&nbsp;<\/sub>\n    <\/td>\n<\/tr>\n<\/table>\n<p>\nHint: Use dynamic programming.\n<\/p>\n<\/blockquote>\n<p>\nIn words:\n<\/p>\n<ul>\n<li>If the array has only two elements, then the result is the average\n    of the two elements.<\/p>\n<li>If the array has more than two elements, then\n    then the result is the sum of the following:<\/p>\n<ul>\n<li>Half the value of the function evaluated on the array with the\n        <i>first<\/i> element deleted\n        and the second parameter incremented by one.<\/p>\n<li>Half the value of the function evaluated on the array with the\n        <i>last<\/i> element deleted\n        and the second parameter incremented by one.<\/p>\n<li>If the second parameter is even,\n        then also add the average of the first and last elements\n        of the original array.\n    <\/ul>\n<\/ul>\n<p>\nThe traditional approach with dynamic programming is to cache intermediate\nresults in anticipation that the values will be needed again later.\nThe na&iuml;ve solution, therefore, would have a cache keyed by\nthe vector and\n<var>k<\/var>.\n<\/p>\n<p>\nMy habit when trying to solve these sorts of recurrence relations\nis to solve the first few low-valued cases by hand.\nThat gives me a better insight into the problem\nand may reveal some patterns or tricks.\n<\/p>\n<table BORDER=\"0\" CELLSPACING=\"0\" CELLPADDING=\"0\" STYLE=\"line-height: 200%\" TITLE=\"f([x sub 0, x sub 1, x sub 2], k) = ...\">\n<tr>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>0<\/sub>,\n               <var>x<\/var><sub>1<\/sub>,\n               <var>x<\/var><sub>2<\/sub>], <var>k<\/var><sub>even<\/sub>)&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    = (<var>x<\/var><sub>0<\/sub> + <var>x<\/var><sub>2<\/sub>) \/ 2 +\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>0<\/sub>, <var>x<\/var><sub>1<\/sub>], <var>k<\/var><sub>even<\/sub> + 1) \/ 2 +\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>1<\/sub>, <var>x<\/var><sub>2<\/sub>], <var>k<\/var><sub>even<\/sub> + 1) \/ 2\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\">&nbsp;<\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    = (<var>x<\/var><sub>0<\/sub> + <var>x<\/var><sub>2<\/sub>) \/ 2 +\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>0<\/sub>, <var>x<\/var><sub>1<\/sub>], <var>k<\/var><sub>odd<\/sub>) \/ 2 +\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>1<\/sub>, <var>x<\/var><sub>2<\/sub>], <var>k<\/var><sub>odd<\/sub>) \/ 2\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\">&nbsp;<\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    = (<var>x<\/var><sub>0<\/sub> + <var>x<\/var><sub>2<\/sub>) \/ 2 +\n      (<var>x<\/var><sub>0<\/sub> + <var>x<\/var><sub>1<\/sub>) \/ 4 +\n      (<var>x<\/var><sub>1<\/sub> + <var>x<\/var><sub>2<\/sub>) \/ 4\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\">&nbsp;<\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    = &frac34;<var>x<\/var><sub>0<\/sub> +\n      &frac12;<var>x<\/var><sub>1<\/sub> +\n      &frac34;<var>x<\/var><sub>2<\/sub>\n    <\/td>\n<\/tr>\n<\/table>\n<p>\nEven just doing this one computation, we learned a lot.\n(Each of which can be proved by induction if you are new to this sort of\nthing, or which is evident by inspection if you&#8217;re handy with math.)\n<\/p>\n<p>\nFirst observation:\nThe function is linear in the array elements.\nIn other words,\n<\/p>\n<table BORDER=\"0\" CELLSPACING=\"0\" CELLPADDING=\"0\" STYLE=\"line-height: 200%\">\n<tr TITLE=\"f(x + y, k) = f(x, k) + f(y,k)\">\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;(<var><b>x<\/b><\/var> + <var><b>y<\/b><\/var>,&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>k<\/var>)&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    =&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;(<var><b>x<\/b><\/var>, <var>k<\/var>) +\n    <var>f<\/var>&thinsp;(<var><b>y<\/b><\/var>, <var>k<\/var>),\n    <\/td>\n<\/tr>\n<tr TITLE=\"f(ax, k) = af(x, k)\">\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;(<var>a<\/var><var><b>x<\/b><\/var>,&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>k<\/var>)&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    = <var>a<\/var>\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;(<var><b>x<\/b><\/var>, <var>k<\/var>).\n    <\/td>\n<\/tr>\n<\/table>\n<p>\nTo save space and improve readability, I&#8217;m using vector notation,\nwhere\nadding two vectors adds the elements componentwise,\nand multiplying a vector by a constant multiples each component.\nThe long-form version of the first of the above equations would be\n<\/p>\n<table BORDER=\"0\" CELLSPACING=\"0\" CELLPADDING=\"0\" STYLE=\"line-height: 200%\" TITLE=\"f([x sub 0 + y sub 0, ..., x sub (n-1) + y sub (n-1)], k) = f([x sub 0, ... x sub (n-1)], k) + f([y sub 0, ... y sub (n-1)], k)\">\n<tr>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>0<\/sub> + <var>y<\/var><sub>0<\/sub>,\n    &hellip;, <var>x<\/var><sub><var>n<\/var>&minus;1<\/sub> +\n              <var>y<\/var><sub><var>n<\/var>&minus;1<\/sub>], <var>k<\/var>) =\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>0<\/sub>, &hellip;, <var>x<\/var><sub><var>n<\/var>&minus;1<\/sub>], <var>k<\/var>) +\n    <var>f<\/var>&thinsp;([<var>y<\/var><sub>0<\/sub>, &hellip;, <var>y<\/var><sub><var>n<\/var>&minus;1<\/sub>], <var>k<\/var>)\n    <\/td>\n<\/tr>\n<\/table>\n<p>\nSince the result is a linear combination\nof the vector elements,\nwe can work just with the coefficients and save ourselves some\ntyping.\n(&#8220;Move to the dual space.&#8221;)\nFor notational convenience, we will write\n<\/p>\n<table BORDER=\"0\" CELLSPACING=\"0\" CELLPADDING=\"0\" STYLE=\"line-height: 200%\">\n<tr>\n<td NOWRAP VALIGN=\"baseline\">\n    &#x27E8;<var>a<\/var><sub>0<\/sub>, &hellip;\n          <var>a<\/var><sub><var>n<\/var>&minus;1<\/sub>&#x27E9; =\n    <var>a<\/var><sub>0<\/sub> <var>x<\/var><sub>0<\/sub> +\n    &#x22ef; +\n    <var>a<\/var><sub><var>n<\/var>&minus;1<\/sub> <var>x<\/var><sub><var>n<\/var>&minus;1<\/sub>\n    <\/td>\n<\/tr>\n<\/table>\n<p>\nSecond observation:\nThe specific value of\n<var>k<\/var> is not important.\nAll that matters is whether it is even or odd,\nand each time we recurse to the next level,\nthe parity flips.\nSo the second parameter is really just a boolean.\nThat greatly reduces the amount of stuff we need to cache,\nas well as increasing the likelihood of a cache hit.\n(The na&iuml;ve version would not have realized that\n<span TITLE=\"f(x, k)\"><var>f<\/var>&thinsp;(<var><b>x<\/b><\/var>, <var>k<\/var>)<\/span>\ncan steal the cached result from\n<span TITLE=\"f(x, k+2)\"><var>f<\/var>&thinsp;(<var><b>x<\/b><\/var>, <var>k<\/var> + 2)<\/span>.)\n<\/p>\n<p>\nOur previous hand computation can now be written as\n<\/p>\n<table BORDER=\"0\" CELLSPACING=\"0\" CELLPADDING=\"0\" STYLE=\"line-height: 200%\" TITLE=\"f([x sub 0, x sub 1, x sub 2], k) = ...\">\n<tr>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>0<\/sub>,\n               <var>x<\/var><sub>1<\/sub>,\n               <var>x<\/var><sub>2<\/sub>], even)&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    = &#x27E8;&amp;frac12, 0, &frac12;&#x27E9; +\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>0<\/sub>, <var>x<\/var><sub>1<\/sub>], odd) \/ 2 +\n    <var>f<\/var>&thinsp;([<var>x<\/var><sub>1<\/sub>, <var>x<\/var><sub>2<\/sub>], odd) \/ 2\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\">&nbsp;<\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    = &#x27E8;&frac12;, 0, &frac12;&#x27E9; +\n    &#x27E8;&frac12;, &frac12;, 0&#x27E9; \/ 2 +\n    &#x27E8;0, &frac12;, &frac12;&#x27E9; \/ 2\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\">&nbsp;<\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    = &#x27E8;&frac12;, 0, &frac12;&#x27E9; +\n    &#x27E8;&frac14;, &frac14;, 0&#x27E9; +\n    &#x27E8;0, &frac14;, &frac14;&#x27E9;\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\">&nbsp;<\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    = &#x27E8;&frac34;, &frac12;, &frac34;&#x27E9;\n    <\/td>\n<\/tr>\n<\/table>\n<p>\nNow that we are working with coefficients, we don&#8217;t need\nto deal with\n<var><b>x<\/b><\/var> any more!\nAll that matters is the length of the vector.\nThis makes our recurrences much simpler:\n<\/p>\n<table BORDER=\"0\" CELLSPACING=\"0\" CELLPADDING=\"0\" STYLE=\"line-height: 200%\">\n<tr TITLE=\"f(2, k) = {half, half} for all k\">\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;(2,&nbsp;\n    <\/td>\n<td><var>k<\/var>)&nbsp;\n    <\/td>\n<td>\n    =&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    &#x27E8;&frac12;, &frac12;&#x27E9;&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    for all <var>k<\/var>.\n    <\/td>\n<\/tr>\n<tr TITLE=\"f(n, even) = {1\/2, 0, ..., 0, 1\/2} + { f(n-1, odd) \/ 2, 0} + {0, f(n-1, odd) \/ 2} for n &gt;= 3\">\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;(<var>n<\/var>,&nbsp;\n    <\/td>\n<td>\n    even)&nbsp;\n    <\/td>\n<td>=&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    &#x27E8;&frac12;, 0, &hellip;, 0, &frac12;&#x27E9; +&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    &#x27E8;<var>f<\/var>&thinsp;(<var>n<\/var>&minus;1, odd) \/ 2, 0&#x27E9; +&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    &#x27E8;0, <var>f<\/var>&thinsp;(<var>n<\/var>&minus;1, odd) \/ 2&#x27E9;&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    for <var>n<\/var> &ge; 3.\n    <\/td>\n<\/tr>\n<tr TITLE=\"f(n, odd) = { f(n-1, even) \/ 2, 0} + {0, f(n-1, even) \/ 2} for n &gt;= 3\">\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;(<var>n<\/var>,&nbsp;\n    <\/td>\n<td>\n    odd)&nbsp;\n    <\/td>\n<td>=&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    &nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    &#x27E8;<var>f<\/var>&thinsp;(<var>n<\/var>&minus;1, even) \/ 2, 0&#x27E9; +&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    &#x27E8;0, <var>f<\/var>&thinsp;(<var>n<\/var>&minus;1, even) \/ 2&#x27E9;&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    for <var>n<\/var> &ge; 3.\n    <\/td>\n<\/tr>\n<\/table>\n<p>\nNow we can carry out a few more hand computations.\n<\/p>\n<table BORDER=\"0\" CELLSPACING=\"0\" CELLPADDING=\"0\" STYLE=\"line-height: 200%\" TITLE=\"f(3, odd) = {f(2, even)\/2, 0} + {0, f(2, even)\/2} = {&frac14;, &frac12;, &frac14;}\">\n<tr>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;(3, odd)&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    =\n    &#x27E8;<var>f<\/var>&thinsp;(2, even) \/ 2, 0&#x27E9; +\n    &#x27E8;0, <var>f<\/var>&thinsp;(2, even) \/ 2&#x27E9;\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\">&nbsp;<\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    =\n    &#x27E8;&frac14;, &frac14;, 0&#x27E9; +\n    &#x27E8;0, &frac14;, &frac14;&#x27E9;\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\">&nbsp;<\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    = &#x27E8;&frac14;, &frac12;, &frac14;&#x27E9;\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;(4, even)&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    = &#x27E8;&frac12;, 0, 0, &frac12;&#x27E9; +\n    &#x27E8;<var>f<\/var>&thinsp;(3, odd) \/ 2, 0&#x27E9; +\n    &#x27E8;0, <var>f<\/var>&thinsp;(3, odd) \/ 2&#x27E9;\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\">&nbsp;<\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    = &#x27E8;&frac12;, 0, 0, &frac12;&#x27E9; +\n    &#x27E8;&#x215b;, &frac14;, &#x215b;, 0&#x27E9; +\n    &#x27E8;0, &#x215b;, &frac14;, &#x215b;&#x27E9;\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\">&nbsp;<\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    = &#x27E8;&#x215d;, &#x215c;, &#x215c;, &#x215d;&#x27E9;\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var>&thinsp;(4, odd)&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    =\n    &#x27E8;<var>f<\/var>&thinsp;(3, even) \/ 2, 0&#x27E9; +\n    &#x27E8;0, <var>f<\/var>&thinsp;(3, even) \/ 2&#x27E9;\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\">&nbsp;<\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    =\n    &#x27E8;&#x215c;, &frac14;, &#x215c;, 0&#x27E9; +\n    &#x27E8;0, &#x215c;, &frac14;, &#x215c;&#x27E9;\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\">&nbsp;<\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    = &#x27E8;&#x215c;, &#x215d;, &#x215d;, &#x215c;&#x27E9;\n    <\/td>\n<\/tr>\n<\/table>\n<p>\nThe interesting thing to observe here is that the recursion\ndoes not branch.\nWhen we reduce the size of the vector by one element,\nthe recursive calls are basically identical.\nWe have to shift the coefficients differently in order\nto build up the results,\nbut the recursive call itself is unchanged.\nThis means that we need to perform only\n<var>n<\/var>&minus;2\nrecursive steps to compute\n<span TITLE=\"f(n, k)\"><var>f<\/var>&thinsp;(<var>n<\/var>, <var>k<\/var>)<\/span>.\n<\/p>\n<p>\nOkay, now that we&#8217;ve studied the problem a bit,\nwe can write the code.\nI&#8217;ll write three versions of the function.\nThe first computes according to the recurrence relation\nas originally written.\nWe use this to verify our calculations.\n<\/p>\n<pre>\nfunction f1(x, k) {\n if (x.length == 2) {\n  return (x[0] + x[1]) \/ 2;\n }\n var term = 0;\n if (k % 2 == 0) {\n  term = (x[0] + x[x.length - 1]) \/ 2;\n }\n return term +\n        f1(x.slice(0,-1), k + 1) \/ 2 +\n        f1(x.slice(1), k + 1) \/ 2;\n}\nImmediate window:\nf1([1,2,3], 0)\n= 4.0\n<\/pre>\n<p>\nOkay, that matches our hand calculations,\n&frac34;&middot;1 + &frac12;&middot;2 + &frac34;&middot;3 = 4.\n<\/p>\n<p>\nNow let&#8217;s do the straight recursive version.\n<\/p>\n<pre>\nfunction dotproduct(a, x)\n{\n var total = 0;\n for (var i = 0; i &lt; a.length; i++) total += a[i] * x[i];\n return total;\n}\nfunction f2a(n, k)\n{\n if (n == 2) return [1\/2, 1\/2];\n var c = new Array(n);\n for (var i = 0; i &lt; n; i++) c[i] = 0;\n if (k % 2 == 0) {\n  c[0] = 1\/2;\n  c[n-1] = 1\/2;\n }\n var inner = f2a(n-1, k+1);\n for (var i = 0; i &lt; n - 1; i++) {\n  c[i] += inner[i] \/ 2;\n  c[i+1] += inner[i] \/ 2;\n }\n return c;\n}\nfunction f2(x, k)\n{\n return dotproduct(f2a(x.length, k), x);\n}\nImmediate window:\nf2([1,2,3], 0)\n= 4.0\n<\/pre>\n<p>\nNotice that there is no dynamic programming involved.\n<b>The hint in the problem statement was a red herring!<\/b>\n<\/p>\n<p>\nFinally, we can eliminate the recursion by iterating\n<code>n<\/code> up from 2.\n<\/p>\n<pre>\nfunction f3(x, k)\n{\n var c = new Array(x.length);\n for (var i = 0; i &lt; x.length; i++) c[i] = 0;\n c[0] = 1\/2;\n c[1] = 1\/2;\n for (var n = 3; n &lt;= x.length; n++) {\n  var carry = 0;\n  for (var i = 0; i &lt; n; i++) {\n   var nextcarry = c[i];\n   c[i] = (carry + c[i]) \/ 2;\n   carry = nextcarry;\n  }\n  if ((k + n + x.length) % 2 == 0) {\n   c[0] += 1\/2;\n   c[n-1] += 1\/2;\n  }\n }\n return dotproduct(c, x);\n}\n<\/pre>\n<p>\nI pulled a sneaky trick here in the place we test whether\nwe are in the even or odd case.\nOfficially, the test should be\n<\/p>\n<pre>\n  if ((k + (x.length - n)) % 2 == 0) {\n<\/pre>\n<p>\nbut since we are interested only in whether the result is\neven or odd,\nwe can just add the components together,\nbecause subtraction and addition have the same effect\non even-ness and odd-ness.\n(If I really felt like micro-optimizing,\nI could fold <code>x.length<\/code> into <code>k<\/code>.)\n<\/p>\n<p>\nOkay, now that we have our code,\nlet&#8217;s interpret the original problem.\n<\/p>\n<p>\nThe expression\n<span TITLE=\"{f(n,k)\/2, 0} + {0, f(n,k)\/2}\">\n&#x27E8;<var>f<\/var>&thinsp;(<var>n<\/var>, <var>k<\/var>) \/ 2, 0&#x27E9; +\n&#x27E8;0, <var>f<\/var>&thinsp;(<var>n<\/var>, <var>k<\/var>) \/ 2&#x27E9;<\/span>\ntakes the vector\n<span TITLE=\"f(n,k)\">\n<var>f<\/var>&thinsp;(<var>n<\/var>, <var>k<\/var>)<\/span>\nand averages it against a shifted copy of itself.\n(The word <i>convolution<\/i> could be invoked here.)\nIf you think of the coefficients as describing the distribution\nof a chemical,\nthe expression calculates the effect of diffusion after one\ntime step according to the simple model\n&#8220;At each time step, each molecule has a 50% chance of moving\nto the left and a 50% chance of moving to the right.&#8221;\n(Since the length of the vector varies with\n<var>n<\/var>,\nI&#8217;ll visualize the vector drawn with center-alignment.)\n<\/p>\n<p>\nThe base case\n&#x27E8;&frac12;, &frac12;&#x27E9;\ndescribes the initial conditions of the diffusion,\nwhere half of the chemicals are on the left and half are\non the right.\nThis is one time step after one unit of the chemical\nwas placed in the center.\nLet&#8217;s get rid of the extra term in the recurrence and focus\njust on the diffusion aspect:\n<\/p>\n<table BORDER=\"0\" CELLSPACING=\"0\" CELLPADDING=\"0\" STYLE=\"line-height: 200%\">\n<tr TITLE=\"d(2) = {half, half}\">\n<td NOWRAP VALIGN=\"baseline\">\n    <var>d<\/var>(2) =\n    &#x27E8;&frac12;, &frac12;&#x27E9;&nbsp;\n    <\/td>\n<\/tr>\n<tr>= 3&#8243;&gt;<\/p>\n<td NOWRAP VALIGN=\"baseline\">\n    <var>d<\/var>(<var>n<\/var>) =\n    &#x27E8;<var>d<\/var>(<var>n<\/var>&minus;1) \/ 2, 0&#x27E9; +\n    &#x27E8;0, <var>d<\/var>(<var>n<\/var>&minus;1) \/ 2&#x27E9;&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\">\n    for <var>n<\/var> &ge; 3.\n    <\/td>\n<\/tr>\n<\/table>\n<p>\nIf you compute these values, you&#8217;ll see that the results are\nawfully familiar.\nI&#8217;ve pulled out the common denominator to avoid the ugly fractions.\n<\/p>\n<table BORDER=\"0\" CELLSPACING=\"0\" CELLPADDING=\"0\" STYLE=\"line-height: 200%\">\n<tr>\n<td NOWRAP VALIGN=\"baseline\" ALIGN=\"center\">\n    1 1\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\" ALIGN=\"center\">\n    entire row divided by 2\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\" ALIGN=\"center\">\n    1 2 1\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\" ALIGN=\"center\">\n    entire row divided by 4\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\" ALIGN=\"center\">\n    1 3 3 1\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\" ALIGN=\"center\">\n    entire row divided by 8\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\" ALIGN=\"center\">\n    1 4 6 4 1\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\" ALIGN=\"center\">\n    entire row divided by 16\n    <\/td>\n<\/tr>\n<tr>\n<td NOWRAP VALIGN=\"baseline\" ALIGN=\"center\">\n    1 5 10 10 5 1&nbsp;&nbsp;\n    <\/td>\n<td NOWRAP VALIGN=\"baseline\" ALIGN=\"center\">\n    entire row divided by 32\n    <\/td>\n<\/tr>\n<\/table>\n<p>\nYup, it&#8217;s Pascal&#8217;s Triangle.\n<\/p>\n<p>\nNotice that the sum across the row equals the amount we are dividing by,\nso that the sum of the row is always 1.\n(This is easy to see from the recurrence relation, since the base\ncase establishes the property that the sum of the coordinates is 1,\nand the recurrence preserves it.)\n<\/p>\n<p>\nThis means that\nwe can calculate the coefficients of\n<var>d<\/var>(<var>n<\/var>)\nfor any value of\n<var>n<\/var> directly,\nwithout having to calculate any of coefficients for smaller values\nof\n<var>n<\/var>.\nThe coefficients are just row\n<var>n<\/var> of Pascal&#8217;s triangle,\n<!-- Mathematical formulas are designed to be pretty, not to be suitable for computation -->\nwhich we know how to compute in\n<var>O<\/var>(<var>n<\/var>)<\/a> time<\/a>.\n<\/p>\n<p>\nNow we can also interpret the extra term we removed at the even steps.\nThat term of the recurrence\nsimulates adding a unit of chemical to the solution\nat every other time step,\nhalf a unit at the left edge and half at the right edge.\nAnd we can calculate these directly in the same way that\nwe calculated the diffusion coefficients,\nsince they basically <i>are<\/i> the diffusion coefficients,\njust with a time and location adjustment.\n<\/p>\n<p>\nI pulled a fast one here.\nMaybe you didn&#8217;t pick up on it:\nI&#8217;m assuming that the two parts of the recurrence\nunrelated to the diffusion behavior\n(the base condition and the extra term at the even steps)\nare independent and can simply be added together.\nYou can show this by noting that given the generalized recurrence\n<\/p>\n<table BORDER=\"0\" CELLSPACING=\"0\" CELLPADDING=\"0\" STYLE=\"line-height: 200%\">\n<tr TITLE=\"f sub G(2, k) = G(2) for all k\">\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var><sub><var>G<\/var><\/sub>(2, <var>k<\/var>)\n    = <var>G<\/var>(2, <var>k<\/var>)\n    <\/td>\n<\/tr>\n<tr TITLE=\"f sub G(n, k) = G(n, k) + { f sub G(n-1, k + 1) \/ 2, 0} + {0, f sub G(n-1, k + 1) \/ 2} for n &gt;= 3\">\n<td NOWRAP VALIGN=\"baseline\">\n    <var>f<\/var><sub><var>G<\/var><\/sub>(<var>n<\/var>, <var>k<\/var>)\n    =\n    <var>G<\/var>(<var>n<\/var>, <var>k<\/var>)\n    +\n    &#x27E8;<var>f<\/var><sub><var>G<\/var><\/sub>(<var>n<\/var>&minus;1, <var>k<\/var> + 1) \/ 2, 0&#x27E9; +\n    &#x27E8;0, <var>f<\/var><sub><var>G<\/var><\/sub>(<var>n<\/var>&minus;1, <var>k<\/var> + 1) \/ 2&#x27E9;\n    for <var>n<\/var> &ge; 3.\n    <\/td>\n<\/tr>\n<\/table>\n<p>\nthen\n<span TITLE=\"f sub (G+H) = f sub G + f sub H\">\n<var>f<\/var><sub><var>G<\/var>+<var>H<\/var><\/sub>\n=\n<var>f<\/var><sub><var>G<\/var><\/sub> +\n<var>f<\/var><sub><var>H<\/var><\/sub><\/span>.\n(As before, induct on the recurrence relations.)\nTherefore, we can solve each of the pieces separately\nand just add the results together.\n<\/p>\n<p>\nIf I had the time and inclination,\nI could work out the solution for\n<\/p>\n<table BORDER=\"0\" CELLSPACING=\"0\" CELLPADDING=\"0\" STYLE=\"line-height: 400%\">\n<tr TITLE=\"C(n,i) + sum for k even, 2 &lt; k &le; n of C(k, i)\/2^k\">\n<td NOWRAP VALIGN=\"baseline\">\n<var>C<\/var>(<var>n<\/var>, <var>i<\/var>) +\n<font SIZE=\"+3\"><var STYLe=\"font-style: normal\">&Sigma;<\/var><\/font><sub><var>k<\/var> even, 2 &lt; <var>k<\/var> &le; n<\/sub>\n<var>C<\/var>(<var>k<\/var>, <var>i<\/var>) \/ 2<sup><var>k<\/var><\/sup>\n    <\/td>\n<\/tr>\n<\/table>\n<p>\nor something similar to that.\nLike I said, I ran out of time and inclination.\nBut if I could come up with an efficient way to compute that\nvalue for all values of\n<var>i<\/var>\nfor fixed\n<var>n<\/var>\n(and I believe there is, I&#8217;m just too lazy to work it out),\nthen I would have an\n<var>O<\/var>(<var>n<\/var>)<\/a> solution\nto the original recurrence.\n<\/p>\n<p>\n(Okay, so the &#8220;If I had the time&#8221; bit is a cop-out, but\nI sort of lost interest in the problem.)<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Here&#8217;s a programming challenge: Evaluate the following recurrence relation efficiently for a given array [x0, &hellip;, xn&minus;1] and integer k. f&thinsp;([x0, x1],&nbsp; k) =&nbsp; (x0 + x1) \/ 2&nbsp; for all k, n = 2.&nbsp; = 3&#8243;&gt; f&thinsp;([x0, &hellip;, xn&minus;1],&nbsp; k) =&nbsp; (x0 + xn&minus;1) \/ 2 +&nbsp; f&thinsp;([x0, &hellip;, xn&minus;2], k + 1) \/ [&hellip;]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-1363","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"<p>Here&#8217;s a programming challenge: Evaluate the following recurrence relation efficiently for a given array [x0, &hellip;, xn&minus;1] and integer k. f&thinsp;([x0, x1],&nbsp; k) =&nbsp; (x0 + x1) \/ 2&nbsp; for all k, n = 2.&nbsp; = 3&#8243;&gt; f&thinsp;([x0, &hellip;, xn&minus;1],&nbsp; k) =&nbsp; (x0 + xn&minus;1) \/ 2 +&nbsp; f&thinsp;([x0, &hellip;, xn&minus;2], k + 1) \/ [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/1363","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=1363"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/1363\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=1363"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=1363"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=1363"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}