\nThere are a variety of message digest algorithms out there,\nMD5 being a particularly popular one.\nThese generate a “message digest” (essentially, a hash)\nso you can detect whether\nsomebody has tampered with a file,\nthe theory being that it’s hard to tamper with a file without\nchanging its hash.\n<\/p>\n
\nBut make sure you record the file size as well as the digest.\n<\/p>\n
\nNot that collisions are necessarily easy to create by mistake.\n(I’ve heard a rumor that the deployment team has seen an MD5 collision,\nbut it’s just a rumor. I have no evidence. Heck, maybe what really\nhappened was that somebody\non the deployment got their MR2 into a car accident…)\n<\/p>\n
\nAnyway, the possibility of a “reset attack” makes collisions\ntrivial to create.\n<\/p>\n
\nHash generators typically operate on a stream.\nThe hash engine maintains some state.\nThe file to be hashed is broken up into chunks, and each chunk\nis combined with the engine’s state variables in some complex way.\nWhen you have passed all the data through the engine, you push a\nbutton on the engine and out pops the hash value (which is typically\na copy of the state variables, or possibly a subset of them).\n<\/p>\n
\nNow suppose somebody came up with a way of “resetting” the engine;\nthat is, returning it to the initial state.\nHere’s how they can\nmake any document match your digest:\n<\/p>\n
\nFirst, create an alternate message and send it through the hash engine.\n<\/p>\n
\nNext, generate the bytes necessary to “reset” the engine.\n<\/p>\n
\nFinally, append the original message.\n<\/p>\n
\nIn other words, the fake file looks like this:\n<\/p>\n
\n[alternate message][garbage][original message]\n<\/pre>\n\nwhere “garbage” is the reset.\n<\/p>\n
\nThis fake file has the same hash as the original message, since\nthe “garbage” resets the hash engine to the initial state,\nat which point the replay of the original message regenerates the\nhash.\n<\/p>\n
\nResult: A file with the same hash as the original, but with\ndifferent content!\n<\/p>\n
\nIn a proper attack, of course, the “alternate message” would be\ncrafted so the garbage and original mesage would be ignored.\nYou might end it with a marker that means\n“Ignore everything after this point.”\n(For HTML, you can just say <NOFRAMES> and everything after\nthat point will be largely ignored by all modern browsers.)\nMany other file types\nencode the expected file length in the header,\nin which case you can append whatever garbage you want without\nhaving any effect.\n<\/p>\n
\nBut if you also store the file size with the hash,\nthen the reset attack fails, because a reset attack always\ngenerates a file bigger than the original.\nTo create a collision, they would have to create a shorter\nalternate message than the original, and then fiddle with the\nextra bytes to get the desired target hash to come out.\nThis is significantly harder than just resetting.\n<\/p>\n
\n(I’m not aware of anybody who has successfully\nbeen able to reset MD5, mind you.\nThis is a protective measure: If somebody figures out how\nto reset MD5, a small bit of work on your side will prevent\nyou from falling victim.)<\/p>\n","protected":false},"excerpt":{"rendered":"
There are a variety of message digest algorithms out there, MD5 being a particularly popular one. These generate a “message digest” (essentially, a hash) so you can detect whether somebody has tampered with a file, the theory being that it’s hard to tamper with a file without changing its hash. But make sure you record […]<\/p>\n","protected":false},"author":1069,"featured_media":111744,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[26],"class_list":["post-39263","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oldnewthing","tag-other"],"acf":[],"blog_post_summary":"
There are a variety of message digest algorithms out there, MD5 being a particularly popular one. These generate a “message digest” (essentially, a hash) so you can detect whether somebody has tampered with a file, the theory being that it’s hard to tamper with a file without changing its hash. But make sure you record […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/39263","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/users\/1069"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/comments?post=39263"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/posts\/39263\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media\/111744"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/media?parent=39263"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/categories?post=39263"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/oldnewthing\/wp-json\/wp\/v2\/tags?post=39263"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}