Support for SAM/QFS offline...

Hierarchical storage management is really useful to offload seldom used data to cheaper media like tape while keeping them in access by a filesystem. SAM/QFS is such a filesystem. Your normal disks are just a cache, the data resides on tapes, magneto-opticals or slow disk drives (Just in case, you want to try it: I wrote a tutorial about the basic installation last year)

But there is a problem: Some operating environments generate a thumbnail to give the user an preview of the file. So you touch each an every file in a directory to generate this little picture.On a normal disk, this isn't a problem, but now think about a hierarchical filesystem, where some data is placed on tapes for example. To access this files, the system does something called "staging":You have to put the tape into the drive, you have to wind, and then you can read the file, then eject it to read the next tape - and this takes time ... a lot of time.

Now imagine an File Management Tool like the Windows Explorer that access all files. You know what happens? Yes, the system tries to stage all files in a directory. We call this effect staging storm. Your autoloader do nothing else than swapping tapes to generate thumbnails.

But there is a less known feature of Windows CIFS. You can declare a file as "offline", thus it just accesses the file when you explicitly try to read it by clicking on it. And here the problem starts: Someone has to tell Windows that this file is offline. The factory-default Samba has no mechanism to do this. This flag isn't supported by generic Samba, thus you can't limit the staging to the file you want to to thumbnail generation. It isn't the fault of Windows, because Samba tells it, that the files on tape aren't offline. So it thinks, they are available like on a hard disk. A countermeasure for this problem has to teach Samba to check for the state of a file in a SAM/QFS filesystem.

My colleague Dirk Nitschke pointed me to an important part of a new patch to the Samba binaries provided by Sun: Patch 119757-17 for SPARC and 119758-17 for x86 doesn't only update the Sun supported version to Samba 3.0.37. It integrates the modification to Samba due to CR 6737005 and PSARC 2009/381, that supports this Offline flag. This flag is set in the communication with a windows client, when the the matching flags in SAM/QFS that designate, that the file was released from the cache and just resides on slower media, is set. When a file carries this flag, the system won't generate an thumbnail, thus there is no need to stage it. To notify the user of this situation, offlined files have a small clock at the left side of the icon (at least in XP)

How does it look in a Windows Explorer? Dirk provided me a screenshot of his SAMQFS/Samba installation. You have to click on it to get a larger version.



When a file is offlined, you will see it by a small clock at the icon for the file. Additional the attribute column shows an O for offline. When you look at the output of sls (the ls of SamFS enabled to show additional SamFS specific properties of a file) the file context.txt is in the mode "offline".

With this patch the combination of SAM/QFS and Samba is much more usable and you don't have to compile your own version from source with a source patch provided by your friendly Sun rep, which isn't supported as a part of the support for the Solaris Operating Environment.