{"id":43503,"date":"2014-12-01T07:00:00","date_gmt":"2014-12-01T07:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/2014\/12\/01\/counting-array-elements-which-are-below-a-particular-limit-value-using-sse\/"},"modified":"2014-12-01T07:00:00","modified_gmt":"2014-12-01T07:00:00","slug":"counting-array-elements-which-are-below-a-particular-limit-value-using-sse","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20141201-00\/?p=43503","title":{"rendered":"Counting array elements which are below a particular limit value using SSE"},"content":{"rendered":"

\nSome time ago, we looked at\n\nhow doing something can be faster than not doing it<\/a>.\nThat is, we observed the non-classical effect of the branch predictor.\nI took the branch out of the inner loop,\nbut let’s see how much further I can push it.\n<\/p>\n

\nThe trick I’ll employ today is using SIMD in order to\noperate on multiple pieces of data simultaneously.\nTake the original program and replace the\ncount­them<\/code>\nfunction with this one:\n<\/p>\n

\nint countthem(int boundary)\n{\n __m128i xboundary = _mm_cvtsi32_si128(boundary);\n __m128i count = _mm_setzero_si128();\n for (int i = 0; i < 10000; i++) {\n  __m128i value =  _mm_cvtsi32_si128(array[i]);\n  __m128i test = _mm_cmplt_epi32(value, xboundary);\n  count = _mm_sub_epi32(count, test);\n }\n return _mm_cvtsi128_si32(count);\n}\n<\/pre>\n
\nNow, this program doesn’t actually use any parallel operations,\nbut it’s our starting point.\nFor each 32-bit value,\nwe load it,\ncompare it agains the boundary value,\nand accumulate the result.\nThe _mm_cmplt_epi32<\/code> function\ncompares the four 32-bit integers in the first parameter\nagainst the four 32-bit integers in the second parameter,\nproducing four new 32-bit integers.\nEach of the new 32-bit integers is 0xFFFFFFFF<\/code>\nif the corresponding first parameter is less than the second,\nor it is 0x00000000<\/cODE> if it is greater than or equal.\n<\/p>\n
\nIn this case, we loaded up the value we care about,\nthen compare it against the boundary value.\nThe result of the comparison is either 32 bits of 0 (for false)\nor 32 bits of 1 (for true),\nso this merely sets test<\/code> equal to\n0xFFFFFFFF<\/code> if the value is less than the boundary;\notherwise\n0x0000000<\/code>.\nSince 0xFFFFFFFF<\/code> is the same as a 32-bit -1<\/code>,\nwe subtract the value so that the count goes up by 1 if the\nvalue is less than the boundary.\n<\/p>\n
\nFinally, we convert back to a 32-bit integer\nand return it.\n<\/p>\n
\nWith this change, the running time drops from 2938 time units\nto 2709, an improvement of 8%.\n<\/p>\n
\nSo far,\nwe have been using only the bottom 32 bits of the 128-bit XMM registers.\nLet's turn on the parallelism.\n<\/p>\n
\nint countthem(int boundary)\n{\n __m128i *xarray = (__m128i*)array;<\/font>\n __m128i xboundary = _mm_set1_epi32<\/font>(boundary);\n __m128i count = _mm_setzero_si128();\n for (int i = 0; i < 10000 \/ 4<\/font>; i++) {\n  __m128i value = _mm_loadu_si128(&xarray[i])<\/font>;\n  __m128i test = _mm_cmplt_epi32(value, xboundary);\n  count = _mm_sub_epi32(count, test);\n }\n __m128i shuffle1 = _mm_shuffle_epi32(count, _MM_SHUFFLE(1, 0, 3, 2));\n count = _mm_add_epi32(count, shuffle1);\n __m128i shuffle2 = _mm_shuffle_epi32(count, _MM_SHUFFLE(2, 3, 0, 1));\n count = _mm_add_epi32(count, shuffle2);<\/font>\n return _mm_cvtsi128_si32(count);\n}\n<\/pre>\n
\nWe take our 32-bit integers and put them in groups of four,\nso instead of thinking of them as 10000 32-bit integers,\nwe think of them as 2500 128-bit blocks,\neach block containing four lanes<\/i>,\nwith each lane holding one 32-bit integers.\n<\/p>\n\n\n\n\n\n\n
<\/td>\n<\/td>\nLane 3<\/td>\nLane 2<\/td>\nLane 1<\/td>\nLane 0<\/td>\n<\/tr>\n
xarray[0]<\/code><\/td>\n<\/td>\narray[3]<\/code><\/td>\narray[2]<\/code><\/td>\narray[1]<\/code><\/td>\narray[0]<\/code><\/td>\n<\/tr>\n
xarray[1]<\/code><\/td>\n<\/td>\narray[7]<\/code><\/td>\narray[6]<\/code><\/td>\narray[5]<\/code><\/td>\narray[4]<\/code><\/td>\n<\/tr>\n
⋮<\/td>\n<\/td>\n⋮<\/td>\n⋮<\/td>\n⋮<\/td>\n⋮<\/td>\n<\/tr>\n
xarray[2499]<\/code><\/td>\n<\/td>\narray[9999]<\/code><\/td>\narray[9998]<\/code><\/td>\narray[9997]<\/code><\/td>\narray[9996]<\/code><\/td>\n<\/tr>\n<\/table>\n
\nNow we can run our previous algorithm in parallel on each lane.\n<\/p>\n\n\n\n\n\n\n\n\n\n\n
<\/td>\n<\/td>\nLane 3<\/td>\nLane 2<\/td>\nLane 1<\/td>\nLane 0<\/td>\n<\/tr>\n
xboundary<\/code><\/td>\n<\/td>\nboundary<\/code><\/td>\nboundary<\/code><\/td>\nboundary<\/code><\/td>\nboundary<\/code><\/td>\n<\/tr>\n
 <\/td>\n<\/tr>\n
test<\/code><\/td>\n<\/td>\narray[3] < boundary<\/code><\/td>\narray[2] < boundary<\/code><\/td>\narray[1] < boundary<\/code><\/td>\narray[0] < boundary<\/code><\/td>\n<\/tr>\n
test<\/code><\/td>\n<\/td>\narray[7] < boundary<\/code><\/td>\narray[6] < boundary<\/code><\/td>\narray[5] < boundary<\/code><\/td>\narray[4] < boundary<\/code><\/td>\n<\/tr>\n
⋮<\/td>\n<\/td>\n⋮<\/td>\n⋮<\/td>\n⋮<\/td>\n⋮<\/td>\n<\/tr>\n
test<\/code><\/td>\n<\/td>\narray[9999] < boundary<\/code><\/td>\narray[9998] < boundary<\/code><\/td>\narray[9997] < boundary<\/code><\/td>\narray[9996] < boundary<\/code><\/td>\n<\/tr>\n
 <\/td>\n<\/tr>\n
count<\/code> = Σ<\/var> −test<\/code><\/td>\n<\/td>\nLane 3 totals<\/td>\nLane 2 totals<\/td>\nLane 1 totals<\/td>\nLane 0 totals<\/td>\n<\/tr>\n<\/table>\n
\nThe xboundary<\/code> variable contains\na copy of the boundary in each of the four 32-bit lanes.\nWe load the values from the array four at a time¹\nand compare them (in parallel) against the boundary,\nthen we tally them (in parallel).\nThe result of the loop is that each lane of count<\/code>\nperforms a count of values for its lane.\n<\/p>\n
\nAfter we complete the loop, we combine the parallel results\nby adding the lanes together. We do this by shuffling the values\naround and performing more parallel adds.\nThe\n_mm_shuffle_epi32<\/code> function lets you rearrange the\nlanes of an XMM register.\nThe _MM_SHUFFLE<\/code> macro lets you specify how you\nwant the shuffle to occur.\nFor example,\n_MM_SHUFFLE(1, 0, 3, 2)<\/code>\nsays that we want lanes 1, 0, 3 then 2 of the original value.\n(You can shuffle a value into multiple destination lanes;\nfor example,\n_MM_SHUFFLE(0, 0, 0, 0)<\/code>\nsays that you want four copies of lane 0.\nThat's how we created xboundary<\/code>.)\n<\/p>\n\n\n\n\n\n\n\n\n\n
<\/td>\n<\/td>\nLane 3<\/td>\nLane 2<\/td>\nLane 1<\/td>\nLane 0<\/td>\n<\/tr>\n
count<\/code><\/td>\n<\/td>\nLane 3 totals<\/td>\nLane 2 totals<\/td>\nLane 1 totals<\/td>\nLane 0 totals<\/td>\n<\/tr>\n
shuffle1<\/code><\/td>\n<\/td>\nLane 1 totals<\/td>\nLane 0 totals<\/td>\nLane 3 totals<\/td>\nLane 2 totals<\/td>\n<\/tr>\n
 <\/td>\n<\/tr>\n
count += shuffle1<\/code><\/td>\n<\/td>\nLane 3 + Lane 1<\/td>\nLane 2 + Lane 0<\/td>\nLane 1 + Lane 3<\/td>\nLane 0 + Lane 2<\/td>\n<\/tr>\n
shuffle2<\/code><\/td>\n<\/td>\nLane 2 + Lane 0<\/td>\nLane 3 + Lane 1<\/td>\nLane 0 + Lane 2<\/td>\nLane 1 + Lane 3<\/td>\n<\/tr>\n
 <\/td>\n<\/tr>\n
count += shuffle2<\/code><\/td>\n<\/td>\nLane 3 + Lane 1 +
\n                                        Lane 2 + Lane 0<\/td>\n		Lane 2 + Lane 0 +
\n                                        Lane 3 + Lane 1<\/td>\n		Lane 1 + Lane 3 +
\n                                        Lane 0 + Lane 2<\/td>\n		Lane 0 + Lane 2 +
\n                                        Lane 1 + Lane 3<\/td>\n<\/tr>\n<\/table>\n
\nAt the end of the shuffling and adding,\nwe have calculated the sum of all\nfour lanes.\n(For\n\nstyle points<\/a>, I put the answer in all the lanes.)\n<\/p>\n
\nThis new version runs in 688 time units,\nor 3.9 times faster than the previous one.\nThis makes sense because we are counting four\nvalues at each iteration.\nThe overall improvement is 4.3×.\n<\/p>\n
\nLet's see if we can reduce the loop overhead by\ndoing some unrolling.\n<\/p>\n
\n#define GETVALUE(n) __m128i value##n = _mm_loadu_si128(&xarray[i+n])\n#define GETTEST(n) __m128i test##n = _mm_cmplt_epi32(value##n, xboundary)\n#define GETCOUNT(n)  count = _mm_sub_epi32(count, test##n)<\/font>\nint countthem(int boundary)\n{\n __m128i *xarray = (__m128i*)array;\n __m128i xboundary = _mm_set1_epi32(boundary);\n __m128i count = _mm_setzero_si128();\n for (int i = 0; i < 10000 \/ 4; i += 4<\/font>) {\n  GETVALUE(0); GETVALUE(1); GETVALUE(2); GETVALUE(3);\n   GETTEST(0);  GETTEST(1);  GETTEST(2);  GETTEST(3);\n  GETCOUNT(0); GETCOUNT(1); GETCOUNT(2); GETCOUNT(3);<\/font>\n }\n __m128i shuffle1 = _mm_shuffle_epi32(count, _MM_SHUFFLE(1, 0, 3, 2));\n count = _mm_add_epi32(count, shuffle1);\n __m128i shuffle2 = _mm_shuffle_epi32(count, _MM_SHUFFLE(2, 3, 0, 1));\n count = _mm_add_epi32(count, shuffle2);\n return _mm_cvtsi128_si32(count);\n}\n<\/pre>\n
\nWe unroll the loop fourfold.\nAt each iteration, we load 16 values from memory,\nand then accumulate the totals.\nWe fetch all the memory values first,\nthen do the comparisons,\nthen accumulate the results.\nIf we had written it as\nGETVALUE<\/code> immediately followed\nby GETTEST<\/code>,\nthen the _mm_cmplt_epi32<\/code>\nwould have stalled waiting for the result\nto arrive from memory.\nBy interleaving the operations,\nwe get some work done instead of stalling.\n<\/p>\n
\nThis version runs in 514 time units,\nan improvement of 33% over the previous version\nand an overall improvement of 5.7×.\n<\/p>\n
\nCan we unroll even further?\nLet's try fivefold.\n<\/p>\n
\nint countthem(int boundary)\n{\n __m128i *xarray = (__m128i*)array;\n __m128i xboundary = _mm_set1_epi32(boundary);\n __m128i count = _mm_setzero_si128();\n for (int i = 0; i < 10000 \/ 4; i += 5<\/font>) {\n  GETVALUE(0); GETVALUE(1); GETVALUE(2); GETVALUE(3); GETVALUE(4);<\/font>\n   GETTEST(0);  GETTEST(1);  GETTEST(2);  GETTEST(3);  GETTEST(4);<\/font>\n  GETCOUNT(0); GETCOUNT(1); GETCOUNT(2); GETCOUNT(3); GETCOUNT(4);<\/font>\n }\n __m128i shuffle1 = _mm_shuffle_epi32(count, _MM_SHUFFLE(1, 0, 3, 2));\n count = _mm_add_epi32(count, shuffle1);\n __m128i shuffle2 = _mm_shuffle_epi32(count, _MM_SHUFFLE(2, 3, 0, 1));\n count = _mm_add_epi32(count, shuffle2);\n return _mm_cvtsi128_si32(count);\n}\n<\/pre>\n
\nHuh?\nThis version runs marginally slower<\/i>,\nat 528 time units.\nSo I guess further unrolling won't help any more.\n(For example, if you unroll a loop so much that\nyou have more live variables than registers,\nthe compiler will need to spill registers to memory.\nThe x86 has eight XMM registers available,\nso you can easily cross that limit.)\n<\/p>\n
\nBut wait, there's still room for tweaking.\nWe have been using\n_mm_cmplt_epi32<\/code> to perform the comparison,\nexpecting the compiler to generate code like this:\n<\/p>\n
\n    ; suppose xboundary is in xmm0 and count is in xmm1\n    movdqu   xmm2, xarray[i] ; xmm2 = value\n    pcmpltd  xmm2, xmm0      ; xmm2 = test\n    psubd    xmm1, xmm2\n<\/pre>\n
\nIf you crack open your Intel manual,\nyou'll see that there is no\nPCMPLTD<\/code> instruction.\nThe compiler intrinsic is emulating the instruction by\nflipping the parameters and using PCMPGTD<\/code>.\n<\/p>\n
\n_mm_cmplt_epi32(x, y) ↔ _mm_cmpgt_epi32(y, x)\n<\/pre>\n
\nBut the PCMPGTD<\/code> instruction writes the result\nback into the first parameter.\nIn other words, it always takes the form\n<\/p>\n
\ny = _mm_cmpgt_epi32(y, x);\n<\/pre>\n
\nIn our case, y<\/code> is xboundary<\/code>,\nbut we don't want to modify xboundary<\/code>.\nAs a result, the compiler needs to introduce a temporary register:\n<\/p>\n
\n    movdqu   xmm2, xarray[i] ; xmm2 = value\n    movdqa   xmm3, xmm0      ; xmm3 = copy of xboundary\n    pcmpgtd  xmm3, xmm2      ; xmm3 = test\n    psubd    xmm1, xmm3\n<\/pre>\n
\nWe can take an instruction out of the sequence by switching to\n_mm_cmpgt_epi32<\/code> and adjusting our logic accordingly,\ntaking advantage of the fact that<\/p>\n
\nx < y ⇔ ¬(x ≥ y) ⇔ ¬(x > y − 1)\n<\/pre>\n
\nassuming the subtraction does not underflow.\nFortunately, it doesn't in our case since boundary<\/code>\nranges from 0 to 10, and subtracting 1 does not put us in any danger\nof integer underflow.\n<\/p>\n
\nWith this rewrite, we can switch to using\n_mm_cmpgt_epi32<\/code>,\nwhich is more efficient for our particular scenario.\nSince we are now counting the values which don't<\/i>\nmeet our criteria,\nwe need to take our final result and subtract it from 10000.\n<\/p>\n
\n#define GETTEST(n) __m128i test##n = _mm_cmpgt<\/font>_epi32(value##n, xboundary1<\/font>)\nint countthem(int boundary)\n{\n __m128i *xarray = (__m128i*)array;\n __m128i xboundary1 = _mm_set1_epi32(boundary - 1);<\/font>\n __m128i count = _mm_setzero_si128();\n for (int i = 0; i < 10000 \/ 4; i += 5<\/font>) {\n  GETVALUE(0); GETVALUE(1); GETVALUE(2); GETVALUE(3); GETVALUE(4);\n   GETTEST(0);  GETTEST(1);  GETTEST(2);  GETTEST(3);  GETTEST(4);\n  GETCOUNT(0); GETCOUNT(1); GETCOUNT(2); GETCOUNT(3); GETCOUNT(4);\n }\n __m128i shuffle1 = _mm_shuffle_epi32(count, _MM_SHUFFLE(1, 0, 3, 2));\n count = _mm_add_epi32(count, shuffle1);\n __m128i shuffle2 = _mm_shuffle_epi32(count, _MM_SHUFFLE(2, 3, 0, 1));\n count = _mm_add_epi32(count, shuffle2);\n return 10000 -<\/font> _mm_cvtsi128_si32(count);\n}\n<\/pre>\n
\nNotice that we have two subtractions which cancel out.\nWe are subtracting the result of the comparison, and then\nwe subtract the total from 10000.\nThe two signs cancel out, and we can use addition for both.\nThis saves an instruction in the return<\/code> because\nsubtraction is not commutative, but addition is.\n<\/p>\n
\n#define GETCOUNT(n) count = _mm_add<\/font>_epi32(count, test##n)\nint countthem(int boundary)\n{\n __m128i *xarray = (__m128i*)array;\n __m128i xboundary1 = _mm_set1_epi32(boundary - 1);\n __m128i count = _mm_setzero_si128();\n for (int i = 0; i < 10000 \/ 4; i += 5) {\n  GETVALUE(0); GETVALUE(1); GETVALUE(2); GETVALUE(3); GETVALUE(4);\n   GETTEST(0);  GETTEST(1);  GETTEST(2);  GETTEST(3);  GETTEST(4);\n  GETCOUNT(0); GETCOUNT(1); GETCOUNT(2); GETCOUNT(3); GETCOUNT(4);\n }\n __m128i shuffle1 = _mm_shuffle_epi32(count, _MM_SHUFFLE(1, 0, 3, 2));\n count = _mm_add_epi32(count, shuffle1);\n __m128i shuffle2 = _mm_shuffle_epi32(count, _MM_SHUFFLE(2, 3, 0, 1));\n count = _mm_add_epi32(count, shuffle2);\n return 10000 +<\/font> _mm_cvtsi128_si32(count);\n}\n<\/pre>\n
\nYou can look at the transformation this way:\nThe old code considered the glass half empty.\nIt started with zero and added 1 each time it found an entry\nthat passed the test.\nThe new code considers the glass half full.\nIt assumes each entry passes the test,\nand it subtracts one each time it finds an element that fails the test.\n<\/p>\n
\nThis version runs in 453 time units,\nan improvement of 13% over the fourfold unrolled version\nand an improvement of 6.5× overall.\n<\/p>\n
\nOkay, let's unroll sixfold, just for fun.\n<\/p>\n
\nint countthem(int boundary)\n{\n __m128i *xarray = (__m128i*)array;\n __m128i xboundary = _mm_set1_epi32(boundary - 1);\n __m128i count = _mm_setzero_si128();\n int i = 0;\n {\n    GETVALUE(0); GETVALUE(1); GETVALUE(2); GETVALUE(3);\n     GETTEST(0);  GETTEST(1);  GETTEST(2);  GETTEST(3);\n    GETCOUNT(0); GETCOUNT(1); GETCOUNT(2); GETCOUNT(3);\n }\n i += 4;<\/font>\n for (; i < 10000 \/ 4; i += 6)<\/font> {\n  GETVALUE(0); GETVALUE(1); GETVALUE(2);\n  GETVALUE(3); GETVALUE(4); GETVALUE(5);<\/font>\n   GETTEST(0);  GETTEST(1);  GETTEST(2);\n   GETTEST(3);  GETTEST(4);  GETTEST(5);<\/font>\n  GETCOUNT(0); GETCOUNT(1); GETCOUNT(2);\n  GETCOUNT(3); GETCOUNT(4); GETCOUNT(5);<\/font>\n }\n __m128i shuffle1 = _mm_shuffle_epi32(count, _MM_SHUFFLE(1, 0, 3, 2));\n count = _mm_add_epi32(count, shuffle1);\n __m128i shuffle2 = _mm_shuffle_epi32(count, _MM_SHUFFLE(2, 3, 0, 1));\n count = _mm_add_epi32(count, shuffle2);\n return 10000 + _mm_cvtsi128_si32(count);\n}\n<\/pre>\n
\nSince 10000 \/ 4 % 6 = 4<\/code>,\nwe have four values that don't fit in the loop.\nWe deal with those values up front,\nand then enter the loop to get the rest.\n<\/p>\n
\nThis version runs in 467 time units,\nwhich is 3% slower than the previous version.\nSo I guess it's time to stop unrolling.\nLet's go back to the previous version which ran faster.\n<\/p>\n
\nThe total improvement we got after all this tweaking\nis speedup of 6.5× over the original jumpless version.\nAnd most of that improvement (5.7×) came from\nunrolling the loop fourfold.\n<\/p>\n
\nAnyway, no real moral of the story today.\nI just felt like tinkering.\n<\/p>\n
\nNotes<\/b>\n<\/p>\n
\n¹ The\n_mm_loadu_si128<\/code>\nintrinsic is kind of weird.\nIts formal argument is a\n__m128i*<\/code>,\nbut since it is for loading unaligned data,\nthe formal argument really should be\n__m128i __unaligned*<\/code>.\nThe problem is that the __unaligned<\/code> keyword\ndoesn't exist on x86 because prior to the introduction of MMX and SSE,\nx86 allowed arbitrary misaligned data.\nTherefore, you are in this weird situation where you have to\nuse an aligned pointer to access unaligned data.\n<\/p>\n
\nBonus chatter<\/b>: Clang at optimization level 3 does autovectorization.\nIt doesn't know some of the other tricks, like converting\nx + 1<\/code>\nto\nx - (-1)<\/code>, thereby saving an instruction and a register.<\/p>\n","protected":false},"excerpt":{"rendered":"
Some time ago, we looked at how doing something can be faster than not doing it. That is, we observed the non-classical effect of the branch predictor. I took the branch out of the inner loop, but let’s see how much further I can push it. The trick I’ll employ today is using SIMD in […]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-43503","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"
Some time ago, we looked at how doing something can be faster than not doing it. That is, we observed the non-classical effect of the branch predictor. I took the branch out of the inner loop, but let’s see how much further I can push it. The trick I’ll employ today is using SIMD in […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/43503","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=43503"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/43503\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=43503"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=43503"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=43503"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}