Taxes: Hierarchical Storage Management

Raymond Chen


One of the taxes I alluded to some time ago when I broached the issues of software development “taxes” is Hierarchical Storage Management. The short description of Hierarchical Storage Management is that it is a way of archiving data transparently. When a file is due for archival, it is transferred to a slower (but less expensive) storage medium, such as magnetic tape, leaving a stub behind.

The stub retains some of the file’s original metadata, such as last-modified time and file size, but none of the original file’s contents are recorded by the stub. If a program tries to open the stub, the original file is “recalled” from tape backup, a process which can take minutes.

Programmatically, you can detect that you stumbled across one of these stubs by checking for the FILE_ATTRIBUTE_OFFLINE file attribute. (Note that this is not the same as Offline Files.) We already saw that Explorer indicates such files with a black clock. The command prompt indicates such files by putting the file size in parentheses. If your program encounters a file with this attribute, it should not open the file unless the user explicitly asked it to do so. Examples of operations that should be suppressed for an offline file in the absence of explicit user indications to the contrary:

  • Auto-preview.
  • Content indexing.
  • Searching.
  • Scanning for viruses.
  • Sniffing file content.

For example, a context menu handler should not open an offline file just to see which context menu options to offer. Right-clicking a file is not a strong enough reason to recall it from tape.

Failing to respect the FILE_ATTRIBUTE_OFFLINE file attribute when performing a search would result in all files accessed during the search being recalled from tape. If left unchecked, this will eventually recall every single file on the system, completely negating the act of archiving the files to tape in the first place!

[Raymond is currently away; this message was pre-recorded.]

Raymond Chen
Raymond Chen

Follow Raymond