A customer wanted to enable NX (also known as Data Execution Prevention, or DEP) and ASLR for some executables and DLLs. There are two ways of doing this: <\/p>\n
\/NXCOMPAT<\/code> and \/DYNAMICBASE<\/code>, and for good measure \/HIGHENTROPYVA<\/code>. <\/li>\n- Build the modules the usual way, and then use the
EDITBIN<\/code> program with those same command line options to enable the features on the files in question. <\/li>\n<\/ul>\nFor reasons the customer didn’t provide (but which I can guess),¹ there are a handful of files that they cannot relink, so they are forced to use the EDITBIN<\/code> approach for those files. <\/p>\nWhat the customer found was that both the linker and the EDITBIN<\/code> approaches seemed to be fine with \/NXCOMPAT<\/code>. But the story was different with the \/DYNAMICBASE<\/code> flag. <\/p>\nSpecifically, the linker approach was creating a larger file due to a new .reloc<\/code> section. The file produced by the EDITBIN<\/code> approach didn’t have a .reloc<\/code> section, and dumping the header reports Relocations stripped<\/b>. Furthermore, when running EDITBIN<\/code>, it generated the ominous warning message, “Warning LNK4259: ‘\/DYNAMICBASE’ is not compatible with ‘\/FIXED’; image may not run.” It appears that linking with the \/DYNAMICBASE<\/code> flag implicitly sets \/FIXED:NO<\/code>, but the EDITBIN<\/code> command doesn’t apply the same behavior, and it doesn’t support the \/FIXED:NO<\/code> command line option. <\/p>\nThe customer had the following questions: <\/p>\n
\n- Are the linker approach and the the
EDITBIN<\/code> approach for enabling \/NXCOMPAT<\/code> equivalent? <\/li>\n<\/ul>\nYes, enabling NX via the linker \/NXCOMPAT<\/code> flag is equivalent to enabling it with EDITBIN<\/code>. In both cases, they set the IMAGE_<\/code>DLLCHARACTERISTICS_<\/code>NX_<\/code>COMPAT<\/code> bit in the IMAGE_<\/code>OPTIONAL_<\/code>HEADER<\/code>‘s DllCharacteristics<\/code>. <\/p>\n\n- Are the linker approach and the the
EDITBIN<\/code> approach for enabling \/DYNAMICBASE<\/code> equivalent? <\/li>\n<\/ul>\nThe answer depends on what color glasses you’re wearing. If you’re wearing kernel-colored glasses<\/a>, then yes, the two approaches are the same because both set the IMAGE_<\/code>DLLCHARACTERISTICS_<\/code>DYNAMIC_<\/code>BASE<\/code> bit in the IMAGE_<\/code>OPTIONAL_<\/code>HEADER<\/code>‘s DllCharacteristics<\/code>. But in reality, they are not the same, because of the behavior noted above with respect to relocations. If you ask the linker for \/DYNAMICBASE<\/code>, it will default to \/FIXED:NO<\/code> because \/DYNAMICBASE<\/code> means “My base address can be moved around”, which is the opposite of \/FIXED<\/code>, which means “My base address cannot change.” <\/p>\nIf you say \/FIXED<\/code>, then the linker does not generate “relocations”, which are the bits of information that describe what adjustments need to be made to your DLL in order to make it happy at its new location. Trying to turn on \/DYNAMICBASE<\/code> with EDITBIN<\/code> on a binary that is \/FIXED<\/code> doesn’t work because EDITBIN<\/code> doesn’t know how to regenerate the .reloc<\/code> section. <\/p>\nIf you want to enable \/DYNAMICBASE<\/code>, then you cannot link with \/FIXED<\/code>. <\/p>\n\n- Are there any adverse consequences of mixing ASLR-enabled DLLs and non-ASLR-enabled DLLs in the same process? <\/li>\n<\/ul>\n
There are no consequences beyond those already stated on the tin. The ASLR-enabled binaries will be subject to ASLR, and the non-ASLR-enabled binaries will not. You can mix and match freely within a process. Just be aware that the non-ASLR-enabled binaries will not be randomly relocated, which means that those binaries will not benefit from the security protections provided by ASLR. <\/p>\n
\n- Are there any adverse consequences of mixing NX-enabled DLLs and non-NX-enabled DLLs in the same process? <\/li>\n<\/ul>\n
The NX setting is process-wide, and the process takes its NX state from the \/NXENABLED<\/code> state of the executable, not from any DLLs<\/a>. It’s yet another one of the module flags that are meaningless for DLLs<\/a>. So mix it up in the DLLs all you want. Nobody will care, because the flag is ignored for DLLs anyway. <\/p>\n\n- Are there any adverse consequences of mixing EXEs and DLLs which use different approaches for enabling NX and\/or ASLR? For example, is there an unexpected interaction between an EXE which enabled ASLR with
EDITBIN<\/code> and a DLL that enabled ASLR with a linker switch? How can I tell whether an EXE or DLL has had its NX or ASLR bit set via the linker as opposed to the EDITBIN<\/code> program? <\/li>\n<\/ul>\nThere is no way to tell how the NX or ASLR attributes were enabled. Using the linker or EDITBIN<\/code> both lead to the same binary. There’s nothing that records who set the bit. As a result, it doesn’t matter how you enabled NX and ASLR. The system behaves the same regardless of how you enabled them. <\/p>\nBonus chatter<\/b>: There is some text on MSDN<\/a> (which got copied into a glossary on TechNet<\/a>) which says <\/p>\nFor a component to support ASLR, all components that it loads must also support ASLR. For example, if A.exe consumes B.dll and C.dll, all three must support ASLR. <\/p><\/blockquote>\n
Nobody is quite sure how that text got into the documentation, since it’s not true. One of the original authors of that article surmises that perhaps what they were trying to say was “In order for a process to take full advantage of ASLR, the executable and all DLLs must support ASLR,” but somehow the message got garbled during editing. <\/p>\n