There are three functions which test very similar things, and sometimes applications pick the wrong one. Here’s the rundown:

• IsThemeActive checks whether visual styles are enabled for the user. This is not an application-specific setting; it tells you whether visual styles are enabled in general. Note that this does not tell you whether the current application is using visual styles.
• IsAppThemed checks whether visual styles are enabled for the current application. Windows may disable visual styles for a specific application (even though they are enabled in general) for compatiblity reasons.
• IsCompositionActive checks whether desktop composition is enabled for the current application. As with visual styles, Windows may disable desktop composition for a specific application (even though it is enabled in general) for compatiblity reasons.

Note that these functions do not answer the question “Is the application using the visual-styles-enabled version of the common controls library?” That question is harder to answer because the decision to use the visual-styles-enabled version of the common controls library is not a process-wide one but is rather made on a window-by-window basis. You can have an application where half of the button controls are the old non-visual-styles version and half of the button controls participate in visual styles. (You may have seen this in action in Explorer, where the OK button on the Run dialog participates in visual styles, yet a button in a shell extension does not.) How can you tell whether a particular button is an old-school button or a fancy new button? I don’t know either.

Bonus emphasis: From the comments, it appears that people have confused “a window was created with the visual-styles-enabled version of the common controls library” with “themes are enabled”. The two are independent concepts. All four combinations are possible. I thought I called this out in the article, but apparently I didn’t call it out clearly enough.