{"id":100175,"date":"2018-11-09T07:00:00","date_gmt":"2018-11-09T22:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/?p=100175"},"modified":"2021-02-13T07:31:42","modified_gmt":"2021-02-13T15:31:42","slug":"20181109-00","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20181109-00\/?p=100175","title":{"rendered":"Gotchas when using linker sections to arrange data, part 2"},"content":{"rendered":"

We saw last time<\/a> that you need to accommodate potential padding between fragments within a section when walking through an array of pointers. Fortunately, it’s a simple matter of skipping over null pointer entries.<\/p>\n

Dealing with padding between fragments when you have a sequence of structures is more complicated, because the amount of padding may not be an exact multiple of the structure size.<\/p>\n

struct THING\r\n{\r\n    const char* name;\r\n    int value;\r\n};\r\n\r\n#pragma section(\"mydata$a\", read)  \r\n__declspec(allocate(\"mydata$a\")) \\\r\n    const THING firstThing{};\r\n\r\n#pragma section(\"mydata$m\", read)  \r\n#define ADD_THING(x, y) \\\r\n__declspec(allocate(\"mydata$m\")) \\\r\n    const THING thing##x{#x, y}\r\n\r\n#pragma section(\"mydata$z\", read)  \r\n__declspec(allocate(\"mydata$z\")) \\\r\n    const THING lastThing{};\r\n\r\n\/\/ file1.cpp\r\nADD_THING(Red, 3);\r\n\r\n\/\/ file2.cpp\r\nADD_THING(Blue, 4);\r\n\r\n\/\/ file3.cpp\r\nADD_THING(Green, 0);\r\n<\/pre>\n
We would be tempted to write<\/p>\n
\/\/ Code in italics is wrong.\r\nvoid LessNaiveRegisterAllTheThings()\r\n{\r\n    auto begin = (uintptr_t)&firstThing + sizeof(firstThing);\r\n    auto end = (uintptr_t)&lastThing;\r\n    for (auto current = begin; current < end;\r\n         current += sizeof(THING)) {\r\n      auto thing = (const THING*)current;\r\n      if (thing->name) RegisterThing(thing->name, thing->value);\r\n    }\r\n}<\/i>\r\n<\/pre>\n
However this will run into trouble if padding is inserted that is not a multiple of sizeof(THING)<\/code>. In that case, advancing by sizeof(THING)<\/code> would eventually cause us to step over some padding bytes as well as the initial bytes of a valid THING<\/code>.<\/p>\n
We will have to walk the pointer past any null bytes until we find the start of a “good” structure.<\/p>\n
This also means that zero cannot be a legitimate value for the first member of a “good” structure, because we wouldn’t be able to figure out whether a zero value is the start of a “good” structure, or whether it’s just padding.<\/p>\n
In the above example, we know that the name is never null, so we can use that as our signal as to whether we have the start of a valid THING<\/code>. If not, then we advance by the alignment of a THING<\/code> and try again.<\/p>\n
void RegisterAllTheThings()\r\n{\r\n    auto begin = (uintptr_t)&firstThing + sizeof(THING);\r\n    auto end = (uintptr_t)&lastThing;\r\n    auto current = begin;\r\n    while (current < end) {\r\n        auto thing = (const THING*)current;\r\n        if (thing->name) {\r\n            RegisterThing(thing->name, thing->value);\r\n            current += sizeof(THING);\r\n        } else {\r\n            current += alignof(THING);\r\n        }\r\n    }\r\n}\r\n<\/pre>\n
A less complicated alternative is to avoid generating structures into ordered segments and just use pointers exclusively.<\/p>\n
#pragma section(\"mydata$a\", read)  \r\n__declspec(allocate(\"mydata$a\")) \\\r\n    const THING* const firstThing = nullptr;\r\n\r\n#pragma section(\"mydata$m\", read)  \r\n#define ADD_THING(x, y, s) \\\r\n    const THING thing##x{#x, y}; \\\r\n__declspec(allocate(\"mydata$m\")) \\\r\n    const THING* const thing##x##ptr = &thing##x;\r\n\r\n#pragma section(\"mydata$z\", read)  \r\n__declspec(allocate(\"mydata$z\")) \\\r\n    const THING* const lastThing = nullptr;\r\n\r\n\/\/ file1.cpp\r\nADD_THING(Red, 3);\r\n\r\n\/\/ file2.cpp\r\nADD_THING(Blue, 4);\r\n\r\n\/\/ file3.cpp\r\nADD_THING(Green, 0);\r\n<\/pre>\n
At this point, we can use the “pointers” pattern.<\/p>\n
void RegisterAllTheThings()\r\n{\r\n    auto begin = (uintptr_t)&firstThing\r\n                 + sizeof(firstThing);\r\n    auto end = (uintptr_t)&lastThing;\r\n    for (auto current = begin; current < end;\r\n         current += sizeof(const THING*)) {\r\n      auto thing = *(const THING* const*)current;\r\n      if (thing) RegisterThing(thing->name, thing->value);\r\n    }\r\n}\r\n<\/pre>\n
For extra style points, you could move the firstThing<\/code> to mydata$b<\/code> and generate the THING<\/code> objects into mydata$a<\/code>. This keeps all the THING<\/code> objects contiguous in memory, which is more cache-friendly. It also keeps them close to the pointer table, which means that they will all page in\/out together. Since this data is probably going to be used only at process startup, putting them all together lets them page out once and stay out.<\/p>\n","protected":false},"excerpt":{"rendered":"
An unexpected null could throw the whole thing out of alignment.<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-100175","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"
An unexpected null could throw the whole thing out of alignment.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/100175","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=100175"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/100175\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=100175"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=100175"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=100175"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}