Rob Mensching, father of Windows Installer XML (WiX), blogs about why managed custom actions are a bad idea. Aaron Stebner follows up by saying (and has said before):

1. “Avoid custom actions entirely if at all possible.
2. “Investigate the WiX build-in custom actions if you find a setup action that is not natively supported in Windows Installer.
3. “If you have to write custom action code, do not use managed code when doing so.”

How do you know what’s natively supported? Those who know me already know what I’m going to say: read the docs. The Windows Installer SDK has some information if you look for it, and you certainly should because setup and servicing cannot be taken for granted. Windows Installer is complex, but a powerful transactional installer. To help disseminate some of this information since it can be sparse, there are a number of good blogs to read about rules, experiences, and guidelines:

• http://blogs.msdn.com/astebner
• http://blogs.msdn.com/heaths
• http://blogs.msdn.com/rflaming/
• http://blogs.msdn.com/windows_installer_team
• http://msmvps.com/blogs/installsite
• http://robmensching.com/blog

There are other things you should do when writing custom actions, and I am working with people throughout the company to define guidelines. In general, besides what Aaron and Rob are saying – along with not using script custom actions – you should consider the following:

• Schedule machine-changing custom actions as deferred custom actions. Especially with UAC, this is very important to support rollback and many other scenarios.
• Developer data-driven custom actions. This makes it easier to service (i.e., patch) resources since you can transform a table and don’t have to reship a custom action. This is especially important to support patch uninstall, since anything a patch added won’t exist when the patch is uninstall and its transform removed from the view.
• In most cases, relate your resources to components in your custom tables and schedule deferred actions based on both the state and the action of each resource’s component. You will find many good examples of this in the WiX sources available on SourceForge.
• Avoid using broadly scoped properties like PATCH or REINSTALL. This can prevent scenarios like slipstream installs where the product and patch are installed together in a single transaction.
• Limit dependencies for custom actions. If you install such dependencies, this limits when your custom actions can be scheduled since your custom action – and installation – will fail without the dependency installed. This is especially important when linking to CRT8 and newer that install as side-by-side native assemblies, and aren’t available on Vista until after the commit phase is complete.

Log. I’ll say it again: log. The Windows Installer engine is quite stable and well secured, so the closest thing available to a crash dump is a verbose log. When run on Windows Installer 3.0 and newer, you can also turn on debug logging. You can use the system Logging policy to set “voicewarmupx” (easy to remember), along with the Debug policy for extra information such as the ever-handy sequence dump which shows you all the information generated to sequence patches. You can also specify logging on the command line or bootstrap developers can call MsiEnableLog(). There are a lot of great posts about parsing and understanding Windows Installer logs.