The dCache HSM Interface

dCache installations, used as a frontend to tertiary storage system, need, at some point, to exchange data this such a system in order to store new, precious files and to retrieve files from the HSM if not yet, or no longer, available on one of the dCache pools. Unfortunately there is no well defined interface for such HSM operations. So the dCache overcomes this problem by calling configurable (dCache external) shell scripts or binaries whenever an HSM store or retrieve operation becomes necessary. The local HSM administrator is responsible for providing this procedure and to make it available and known to the dCache. This small writeup defines the way dCache will call such an external method.

Defining the HSM interface.

Each individual pool, which is expected to exchange data with an HSM, has to define a dCache external method to flush/fetch datasets into/from one or more connected HSM's. The command decribed below has either to be given in the command line interface of the corresponding pool while the pool is active (don't forget so 'save') or may be added to the pool setup file commands prior to starting the pool.
     Syntax : hsm set <hsmName> -command=<fullPathToExternalCommand>
     
     Example :
         
	     hsm set osm -command=/usr/d-cache/jobs/osm-hsmcp.sh
	 or
	     hsm set enstore -command=/usr/d-cache-deployment/jobs/real-enstore.sh
  
The external method, which might be a shell script or a binary, is called by the dCache with a set of positional arguments (see below). In addition, options may be specified which are appended to the regular argument list on calling the external method.
     Syntax : hsm set <hsmName> -<key>=<value>
     
     Example :
         
	     hsm set osm -command=/usr/d-cache/jobs/osm-hsmcp.sh
	     hsm set osm -pnfs=/pnfs/desy.de -somethingElse=true
  
  
This will result in excuting the following command line whenever a file has to be exchanged with an HSM.
    /usr/d-cache/jobs/osm-hsmcp.sh put|get <pnfsId> <LocalFilename> \
        -si=<See Below> \
        -pnfs=/pnfs/desy.de   \
        -somethingElse=true
  

Calling sequence

The external script or binary is launed with 3 positional arguments and at least one option (-si=<storageInfo>). Additonial options may follow if defined so with the hsm set pool command. Arguments and options are separated by at least on blank character.
    Syntax :
    
        <binary> put|get <pnfsid> <localFileName>   \
               -si=<storageInfo> [more options]
  

The put|get argument determines the data transfer direction seen from the HSM. put means, that data has to be stored into the HSM while get means it has be fetched out of the HSM.

The <storageInfo> option is a collection of key value pairs, separated by semicola. All these values are derived from the pnfs database. The possible keys slightly differ, depending on which HSM is addressed. The order of the key value pairs is not determined and may vary between calls. The -si= string shouldn't contain blank TAB or newline characters.

Mandatory StorageInfo keys

KeyMeaning
sizeSize of the file in bytes
newfalse if file already in the dCache
storedtrue if file already stored in the HSM
sClassHSM depended. Used by the PoolManager for pool attraction
cClassParent Director tag (cacheClass). Used by the PoolManager for pool attraction. May be '-'
hsmStorage Manager name (enstore/osm). Can be overwritten by parent directory tag (hsmType).

Optional StorageInfo keys but used by all HSM's

KeyMeaning
flag-lSize of the file (if size exceeds 2G)
flag-s* if file is defined sticky
flag-cCRC value (currently 1:<hexAdler32>

Enstore specific

KeyMeaning
groupStorage Group (e.g. cdf,cms ...)
familyFile family (e.g. sgi2test,h6nxl8, ...)
bfidBitfile Id (GET only) (e.g. B0MS105746894100000)
volumeTape Volume (GET only) (e.g. IA6912)
locationLocation on tape (GET only) (e.g. : 0000_000000000_0000117)

OSM specific

KeyMeaning
storeOSM store (e.g. zeus,h1, ...)
groupOSM Storage Group (e.g. h1raw99, ...)
bfidBitfile Id (GET only) (e.g. 000451243.2542452542.25424524)

There might be more key values pairs which are used by the dCache internally and which should not affect the behaviour of the hsm copy script.


Return codes

Return Code Meaning Pool Behaviour
Into HSMFrom HSM
30 <= rc < 40 User defined Deactivates request Reports Problem to PoolManager
41 No Space Left on device Pool Retries Disables Pool
Reports Problem to PoolManager
42 Disk Read I/O Error
43 Disk Write I/O Error
All other   Pool Retries Reports Problem to PoolManager

Special Cases and exceptions

Reading vers. Writing HSM files

When fetching a file from an HSM, the command line contains sufficient information about the location of the dataset within the HSM to get the file. No additional interaction with pnfs is needed. So pnfs doesn't need to be mounted on read-only pools.

This is different for storing files into an HSM. As a return from the actual HSM put operation, some data has to be stored in pnfs. Currently this has to be done directly by the corresponding external HSM script. So, other then for read pools, write pools still need to have pnfs mounted.

A future approache will be to transfer the necessay HSM information from the HSM copy script into the dCache using STDOUT. The dCache subsequently performs the necessary pnfs store operation through the PnfsManager.

Precious files are removed from Pnfs

In case a precious file is removed from pnfs BEFORE the hsmcopy-Script (osmcp.sh or real-encp.sh) is called, the copy on disk is removed and the hsmcopy-Script is not called.

If the file is removed while the hsmcopy-Script is active, the script will encounter an error when writing HSM data into the various pnfs layers. In this case it's recommended to return an error code in the 30-39 range to have the request deactivated. So manual intervention is needed to get the situation cleaned up but no attempt is made by the dCache to get the corresponding dataset stored into the HSM again.


Author : Patrick Fuhrmann <patrick.fuhrmann@desy.de>
$Id: dCache-Hsm-Interface.html,v 1.1 2005/04/29 12:38:59 patrick Exp $