{"id":30401,"date":"2022-04-26T15:00:14","date_gmt":"2022-04-26T15:00:14","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/cppblog\/?p=30401"},"modified":"2022-05-19T14:52:36","modified_gmt":"2022-05-19T14:52:36","slug":"openmp-task-support-for-c-in-visual-studio","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/cppblog\/openmp-task-support-for-c-in-visual-studio\/","title":{"rendered":"OpenMP Task Support for C++ in Visual Studio"},"content":{"rendered":"

In our previous blog post<\/a> about OpenMP support in Visual Studio 2019 version 16.10, we announced support for the -openmp:llvm<\/strong> switch to enable the compiler to target LLVM\u2019s OpenMP runtime library on x86, x64, and arm64 platforms. In that release, -openmp:llvm<\/strong> included some correctness fixes and support for unsigned indices in parallel for loops, but otherwise only supported the same OpenMP 2.0 features as -openmp<\/strong>.<\/p>\n

-openmp:llvm support for tasks<\/h2>\n

Starting with Visual Studio 2022 version 17.2<\/a>, we have added support for the first major new OpenMP feature specific to the -openmp:llvm<\/strong> flag: the task<\/code> directive as defined by the OpenMP 3.1 standard, including the accompanying if<\/code>, private<\/code>, firstprivate<\/code>, shared<\/code>, default<\/code>, untied<\/code>, mergeable<\/code>, and final<\/code> clauses, and the associated taskwait<\/code> and taskyield<\/code> directives. The compiler does not yet support task<\/code> clauses added in later versions of the OpenMP standard.<\/p>\n

The OpenMP task<\/code> directive is used to specify a unit of work that can be done in parallel by any thread. A task executes once when a thread becomes available, instead of executing once per thread as a parallel region would. Creating tasks is a useful alternative to the OpenMP parallel for<\/code> directive when the number of iterations is not known at the start of the calculation, such as when operating on a linked list or implementing a recursive algorithm.<\/p>\n

The following is a simple example of using the task<\/code> directive to sum the elements of an array by splitting the array into pieces and having each task sum one piece.<\/p>\n

#include <stdio.h>\r\n\r\nint sumArrayImpl(int* arr, int arrayLength) {\r\n  if (arrayLength == 1)\r\n     return arr[0];\r\n  if (arrayLength == 0)\r\n     return 0;\r\n\r\n  int left, right;\r\n  int halfLength = arrayLength \/ 2;\r\n  #pragma omp task default(none) firstprivate(arr, halfLength), shared(left) final(halfLength >= 2)\r\n  {\r\n     left = sumArray(arr, halfLength);\r\n  }\r\n  #pragma omp task default(none) firstprivate(arr, halfLength, arrayLength) shared(right) final(halfLength >= 2)\r\n  {\r\n     right = sumArray(arr + halfLength, halfLength + (arrayLength % 2));\r\n  }\r\n  #pragma omp taskwait\r\n     return left + right;\r\n}\r\n\r\nint sumArray(int* array, int arrayLength)\r\n   #pragma omp parallel\r\n   {\r\n      #pragma omp single\r\n      { \r\n         printf(\"Sum = %dn\", sumArrayImpl(array, arrayLength));\r\n      }\r\n   }\r\n}<\/code><\/pre>\n
Although this example does not do enough work for each element to see a speedup over a sequential for loop, and is consistent enough that a parallel for<\/code> directive could accomplish the same degree of paralellism, it has a similar form to more complicated examples that would see a speedup and illustrates many of the task<\/code> directive clauses.<\/p>\n
The private<\/code>, firstprivate<\/code>, shared<\/code>, and default<\/code> clauses specify the scope of variables used in a task, following a similar pattern to the same clauses on the parallel<\/code> directive. Note that marking a pointer as firstprivate<\/code> means that each task will have its own copy of the pinter. The underlying array is still shared across all tasks.<\/p>\n
The final<\/code> clause prevents creating an excessive number of tasks by causing any further child tasks to be executed sequentially when the clause\u2019s condition is true. An if<\/code> clause, conversely, causes the current task<\/code> region to execute sequentially, but it can still create child tasks that execute in parallel. The taskwait<\/code> directive allows for synchronization between tasks by waiting until a task\u2019s children complete before continuing.<\/p>\n
A few task<\/code> clauses and an additional task-related directive are missing from this example. The taskyield<\/code> directive allows the runtime to suspend a task\u2019s execution in order to execute other tasks, and is useful when a task may need to wait for some other work to complete. The mergeable<\/code> and untied<\/code> clauses on the task<\/code> directive are optimization hints. An untied<\/code> task that yields may resume on any thread, instead of only resuming on the thread that spawned the task. A mergeable<\/code> task allows the runtime to reuse the data environment of its parent for the child task.<\/p>\n
Now, let’s take a look at an example that shows the usefulness of task<\/code> in a scenario where parallel for<\/code> is insufficient. For this example, we’ll be using a data structure designed for storing words called a Trie tree. In a Trie tree, each word is stored as a path through the tree, terminating in a node marked with sequenceEnd == true<\/code>. A node with sequenceEnd == true<\/code> may still have children if it is a substring of another word. For example, a Trie tree containing the words “car”, “care”, and “cars” would start with a c<\/code> node, which has an a<\/code> as a child, which in turn has an r<\/code> as a child. The r<\/code> node would be marked as an end node and also have two children, an e<\/code> leaf and an s<\/code> leaf, both also marked as terminating nodes, like so:<\/p>\n
c\r\n \r\n  a\r\n   \r\n    r*\r\n   \/ \r\n  e*  s*   <\/code><\/pre>\n
A parallel for<\/code> could not traverse a tree like this because there is no random access iterator, but a tree traversal can still take advantage of parallelism by creating a task for each node visited. Consider the following code for counting and printing out all of the words in a trie tree:<\/p>\n
struct TrieNode {\r\n   TrieNode* parent;\r\n   std::unordered_map<char, TrieNode*> children;\r\n\r\n   bool sequenceEnd;\r\n   char currentChar;\r\n\r\n   void Print();\r\n   void PrintAllWordsWithSubstring();\r\n\r\n   ~TrieNode();\r\n};\r\n\r\nvoid TrieNode::Print() {\r\n  if (parent) {\r\n     parent->Print();\r\n     std::cout << currentChar;\r\n  }\r\n}\r\n\r\nint totalWords;\r\n\r\nvoid TrieNode::PrintAllWordsWithSubstring() {\r\n   #pragma omp task\r\n   {\r\n      for (auto child : this->children) {\r\n         child.second->PrintAllWordsWithSubstring();\r\n      }\r\n\r\n      if (this->sequenceEnd) {\r\n          #pragma omp critical\r\n          {\r\n             this->Print();\r\n             std::cout << std::endl;\r\n          }\r\n\r\n          #pragma omp atomic\r\n             ++totalWords;\r\n      }\r\n   }\r\n}\r\n\r\nvoid PrintAllWords(TrieNode* root) {\r\n   totalWords = 0;\r\n\r\n   #pragma omp parallel\r\n   {\r\n      #pragma omp single\r\n      root->PrintAllWordsWithSubstring();\r\n   }\r\n}<\/code><\/pre>\n
In this example, the PrintAllWordsWithSubstring<\/code> member function recursively creates a new task for each node. The this<\/code> pointer is implicitly firstprivate<\/code> inside the task<\/code> region. Choosing the correct data-sharing attributes for variables is especially important for task<\/code> because a task’s execution is deferred instead of executed immediately, as a parallel<\/code> region is. As a result, the function that creates the task may return before the task<\/code> region is executed and any shared<\/code> variables on the stack may fall out of scope. If the this<\/code> pointer was shared, a task in a member function couldn’t reliably access member variables. To see the difference more clearly, consider the following broken alternative implementation of TrieNode::PrintAllWordsWithSubstring<\/code>:<\/p>\n
void TrieNode::PrintAllWordsWithSubstring() {\r\n   for (auto child : this->children) {\r\n      #pragma omp task shared(child)\r\n      {\r\n         \/\/ This line will cause an Access Violation.\r\n         child.second->PrintAllWordsWithSubstring();\r\n      }\r\n   }\r\n\r\n   if (this->sequenceEnd) {\r\n      this->Print();\r\n      std::cout << std::endl;\r\n\r\n      #pragma omp atomic\r\n         ++totalWords;\r\n   }\r\n}<\/code><\/pre>\n
If a variable is only read and never written to inside a parallel<\/code> region, marking it as shared<\/code> doesn’t change the end result. No writes will occur during the execution of the parallel<\/code> region, so all threads will see the same value. However, code outside of a task<\/code> region may execute concurrently with the execution of a task<\/code>. In the flawed implementation above, by the time the recursive call to PrintAllWordsWithSubstring(child.second)<\/code> is made, the iterator will likely already have reached the end of children<\/code> and child.second<\/code> will no longer have a valid value.<\/p>\n
Our OpenMP Plans<\/h2>\n
As of 17.2, all of the OpenMP 2.5 standard is supported, as well as tasks and parallel for loops with unsigned indices from the OpenMP 3.1 standard. We have started the long process to support newer versions of the OpenMP standard. Our ultimate goal is to support the most recent OpenMP standard by leveraging LLVM’s OpenMP runtime, but this will take time. Our next step for OpenMP will be to support the remaining features added in the OpenMP 3.1 standard. Which features are added first will depend on your feedback. We would love to hear from you which specific OpenMP 3.1 features you would like to see first.<\/p>\n
Feedback<\/h2>\n
We encourage you to try out this update in the latest Visual Studio 2022 version 17.2 Preview<\/a>. If you encounter a correctness issue in code generated with the -openmp:llvm<\/strong> switch or bugs in the libomp140 DLLs shipped with Visual Studio, please let us know. We can be reached via the comments below, via twitter (@visualc<\/a>) or via Developer Community<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"
The MSVC compiler adds support for OpenMP tasks with the \/openmp:llvm flag.<\/p>\n","protected":false},"author":18811,"featured_media":35994,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1,3921],"tags":[],"class_list":["post-30401","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cplusplus","category-openmp"],"acf":[],"blog_post_summary":"
The MSVC compiler adds support for OpenMP tasks with the \/openmp:llvm flag.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts\/30401","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/users\/18811"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/comments?post=30401"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/posts\/30401\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/media\/35994"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/media?parent=30401"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/categories?post=30401"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/cppblog\/wp-json\/wp\/v2\/tags?post=30401"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}