Alternate Data Streams; the perils thereof

We set up a Windows file share with "append-only" permissions, meaning anyone in the group in question can add files or directories on this share, but not change or delete anything already there. This works, more or less, but with some annoying artifacts - mostly due to the point-and-clickiness of Windows as a file management interface.

For example, creating a new directory is easy enough, but giving it the proper name is a separate operation, which means modifying the directory, which is a no-no.

These problems are easy enough to work around. All you have to do is create any structures you want locally and then copy them over as a whole.

Recently we've started to notice a weird error cropping up with certain files, particularly ZIP and EXE files and the like. Copying the file seems to work fine, until the very end of the operation, when a useful error like this pops up: "Cannot copy file. The operation completed successfully."

An inexplicable error like this hints at some sinister misdesign deep down in the system, and we were stumped until we realized that the troublesome files had been copied from another share. Right-clicking on the file and clicking the Unblock button fixed the problem.

A quick check with ProcMon showed that the operation was done in two steps (see above...) and it was the second step that failed. It consisted of a call to CreateFile() on the file name MyFile.zip:Zone.Identifier on the destination share (with the relevant file name substituted for MyFile.zip, of course).

Of course this fails. But what is going on here? It turns out that the Windows XP SP2 Attachment Manager stores its information about "insecure" files (to know when to display its "The publisher could not be verified. Are you sure you want to run this software?" warning) as a Zone ID in an Alternate Data Stream in the file in question. ADS is the NTFS answer to the old MacOS HFS "forks", which MacOS X is trying its best to retire entirely for practical reasons. I knew they existed in NTFS but I had never seen them used before, and I didn't even know what they were called.

This can be verified by running a command like more < MyFile.zip:Zone.Identifier, which will produce output like this:

[ZoneTransfer]
ZoneId=3

Indeed, unblocking the file removes this ADS which is what makes it possible to copy the file.

Of course, there is still no excuse for the silly error message above, but at least it makes it painfully obvious that the whole metadata solution was rather hastily thrown together rather than carefully designed, implemented and tested. But who is surprised by that?

I put together a shell extension that provides a submenu in Windows Explorer, wherein you can see and change the Zone ID for a file. Find the installer for this extension attached to this page.

There is still a strange issue that I haven't solved yet. If I produce a blocked EXE file and try to run it, I get the "publisher is not verified" warning, with a checkbox to "Always ask before opening this file". If i uncheck the checkbox and press Run, everything is fine. However, if I press Cancel, remove the Zone Identifier and try to run the program again, the same dialog comes up, but without the checkbox. So if the file is safe, I can't mark it as such - but if it isn't, I can? Maybe it's some weird cache effect, though. Who knows?

AttachmentSize
FileZoneHandler Installer (32-bit)2.06 MB
FileZoneHandler Installer (64-bit)1.55 MB

Comments

Thanks much for this post. I recently had an issue where I was trying to use Windows Server 2003's built-in ZIP file handling to extract the contents of a .zip file, but when it got to a .msi file in the archive, it'd just give a useless error message saying it couldn't copy the file, with no button to say "go ahead despite the security risk". Took a lot of searching to learn that it was necessary to Unblock the ZIP file in order to be able to extract the MSI from it.

Your error message definitely beats the one I got, though, and may be Microsoft's most unhelpful ever (which is saying a lot).