{"id":40833,"date":"2004-01-29T07:00:00","date_gmt":"2004-01-29T07:00:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/oldnewthing\/2004\/01\/29\/integer-overflow-in-the-new-operator\/"},"modified":"2004-01-29T07:00:00","modified_gmt":"2004-01-29T07:00:00","slug":"integer-overflow-in-the-new-operator","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/oldnewthing\/20040129-00\/?p=40833","title":{"rendered":"Integer overflow in the new[] operator"},"content":{"rendered":"

Integer overflows are becoming a new security attack vector.\n\nMike Howard’s article discusses some of the ways you can protect\nyourself against integer overflow attacks<\/a>.<\/p>\n

\nOne attack vector he neglects to mention is integer overflow\nin the new[] operator. This operator performs an implicit multiplication\nthat is unchecked:\n<\/p>\n

\nint *allocate_integers(int howmany)\n{\n    return new int[howmany];\n}\n<\/pre>\n
\nIf you study the code generation for this, it comes out to\n<\/p>\n
\n  mov  eax, [esp+4] ; eax = howmany\n  shl  eax, 2       ; eax = howmany * sizeof(int)\n  push eax\n  call operator new ; allocate that many bytes\n  pop  ecx\n  retd 4\n<\/pre>\n
\nNotice that the multiplication by sizeof(int) is not checked\nfor overflow. Somebody can trick you into under-allocating\nmemory by passing a value like howmany = 0x40000001.\nFor larger structures, multiplication overflow happens sooner.\n<\/p>\n
\nLet’s look at a slightly longer example:\n<\/p>\n
\nclass MyClass {\npublic:\n  MyClass(); \/\/ constructor\n  int stuff[256];\n};\nMyClass *allocate_myclass(int howmany)\n{\n  return new MyClass[howmany];\n}\n<\/pre>\n
\nThis class also contains a constructor,\nso allocating an array of them involves\ntwo steps: allocate the memory, then\nconstruct each object.\nThe allocate_myclass function compiles\nto this:<\/p>\n
\n  mov  eax, [esp+4] ; howmany\n  shl  eax, 10      ; howmany * sizeof(MyClass)\n  push esi\n  push eax\n  call operator new ; allocate that many bytes\n  mov  esi, eax\n  test esi, esi\n  pop  ecx\n  je   fail\n  push OFFSET MyClass::MyClass\n  push [esp+12]     ; howmany\n  push 1024         ; sizeof(MyClass)\n  push esi          ; memory block\n  call `vector constructor iterator`\n  mov  eax, esi\n  jmp  loop\nfail:\n  xor  eax, eax\ndone:\n  pop  esi\n  retd 4\n<\/pre>\n
\nThis function does an unchecked multiplication of\nthe size, then tries to allocate that many bytes,\nthen tells the vector constructor iterator to\ncall the constructor (MyClass::MyClass) that many\ntimes.\n<\/p>\n
\nIf somebody tricks you into calling\nallocate_myclass(0x200001), the multiplication\noverflows and only 1024 bytes are allocated.\nThis allocation succeeds, and then the vector\nconstructor tries to initialize 0x200001 of\nthose items, even though in reality only one\nof them got allocated.  So you walk off the end\nof the memory block and start corrupting memory.\n<\/p>\n
\nThat’s a bad thing.\n<\/p>\n
\nTo protect against this, you can wrap an integer\noverflow check around the array allocation.\n<\/p>\n
\ntemplate<typename T>\nT* NewArray(size_t n)\n{\n  if (n <= (size_t)-1 \/ sizeof(T))\n    return new T[n];\n  \/\/ n is too large - act as if we\n  \/\/ ran out of memory\n  return NULL;\n}\n<\/pre>\n
\nNote: If you use a throwing “new”, then replace\nthe “return NULL” with an appropriate throw.\n<\/p>\n
\nYou can now use this template to allocate\narrays in an overflow-safe manner.\n<\/p>\n
\nMyClass *allocate_myclass(int howmany)\n{\n  return NewArray<MyClass>(howmany);\n}\n<\/pre>\n
\nThis generates the following code:<\/p>\n
\n  push edi\n  mov  edi, [esp+8] ; howmany\n  cmp  edi, 4194303 ; overflow?\n  ja   overflow<\/font>\n  mov  eax, edi\n  shl  eax, 10\n  push esi\n  push eax\n  call operator new\n  mov  esi, eax\n  test esi, esi\n  pop  ecx\n  je   failed\n  push OFFSET MyClass::MyClass\n  push edi\n  push 1024\n  push esi\n  call\n  call `vector constructor iterator`\n  mov  eax, esi\n  jmp  done\nfailed:\n  xor  eax, eax\ndone:\n  pop  esi\n  jmp  exit\noverflow:\n  xor     eax, eax\nexit:\n  pop  edi\n  retd 4\n<\/pre>\n
\nNotice the new code that checks for a possible\ninteger multiplication overflow.\n<\/p>\n
\nBut how could you get tricked into an overflow situation?\n<\/p>\n
\nThe most common way of doing this is by reading the value out\nof a file or some other storage location. For example,\nif your code is parsing a file that has a section whose\nformat is “length followed by data”,\nsomebody could intentionally put an overflow-inducing value\ninto the “length” field, then get somebody else to try\nto load the file.\n<\/p>\n
\nThis is particularly dangerous if the filetype is something\nthat is generally considered “not dangerous”, like a JPG.<\/p>\n","protected":false},"excerpt":{"rendered":"
Integer overflows are becoming a new security attack vector. Mike Howard’s article discusses some of the ways you can protect yourself against integer overflow attacks. One attack vector he neglects to mention is integer overflow in the new[] operator. This operator performs an implicit multiplication that is unchecked: int *allocate_integers(int howmany) { return new int[howmany]; […]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[25],"class_list":["post-40833","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-code"],"acf":[],"blog_post_summary":"
Integer overflows are becoming a new security attack vector. Mike Howard’s article discusses some of the ways you can protect yourself against integer overflow attacks. One attack vector he neglects to mention is integer overflow in the new[] operator. This operator performs an implicit multiplication that is unchecked: int *allocate_integers(int howmany) { return new int[howmany]; […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/40833","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=40833"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/40833\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=40833"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=40833"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=40833"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}