Request list : dCache 1.5.2

Content

TopicStatus
dccp : wildcards in argument list Testing
dCap : opening directory path in url syntax must return error To Be Done
dCap : Kerberized 'create' with none url syntax Possibly never (see comment)
dCap : selection of dcache.conf tunnels Testing
Overbooked Pools : bug fix and rescan TBD
HSM connection : handling zero size files Done
CRC checking In progress
Replica Manager issues In progress
Pnfs not mounted on pool nodes Done for ReadOnly
Simple text tables in addition to .html view (wget) In progress
Assigning unique ID to requests To Be Done

dccp : wildcards in argument list

Description

Using wildcards in dccp argument list.

Comment

A new application mdccp is created which allows to use wildcards in source and destination argument lists. File are processed in the given sequence and not in parallel.

Status

In test phase.

dCap : opening directory path in url syntax must return error

Description

Opening a directory path, using the url style syntax, should return an error. Right now, this is not the case.

Status

TBD

dCap : Kerberized 'create' with none url syntax

Description

It would possibly be desirable to kerberize the create operation on a mounted 'pnfs' the same way it is done with the url like syntax.

Comment

This would require to mount pnfs ReadOnly on all clients and to perform all modification operations through the server. It's not clear if we really want to do that.

Status

Not scheduled yet.

dCap : selection of dcache.conf tunnels

Description

Entries in the (pnfs) dcache.conf file will get keys (tags). dccp resp. the library may specify a key and with that select a set of entries to be used.

Status

Testing.

Overbooked Pools : bug fix and rescan

Description

Under conditions which are not yet clear, heavily used pools tend to increase the used filespace beyond its defined limits. Consequently, sooner or later, this leads to 'filesystem full' errors.

Comment

Its very likely that this problem is related to a misbehaviour or mistreatment of the external 'osmcp/encp' scripts. One 'code candidate' is identified and will be fixed. In addition, we need to implement a rescan of the repository to avoid restarting the pool whenever the problem becomes painful.

Status

TBD.

HSM connection : handling zero size files

Description

Zero size files are not handled correctly when an HSM is involved. The file becomes precious but no attempt is made to write it into the HSM. Consequently an error is produced when the dCache tries to get it back from the HSM.

Comment

DESY and FERMI have different ideas how to handle those files. Therefore the behaviour is configurable by the pool command line option flushZeroSizeFiles
  • -flushZeroSizeFiles=yes The HSM backend/flush/restore scripts are called for zero size files as for any other regular file.
  • -flushZeroSizeFiles=noThe HSM backend/flush/restore are never called for zero size files but their behaviour is simulated.
In both cases, zero size files are now handled correctly.

Status

Done

CRC checking

Description

In order to ensure data integrity a Check Sum Module (csm) has been added to the data pools and the dCap library. The idea is that at different points in the lifetime of a file a checksum is calculated and, depending on the configuration different action may be triggered if calculated and stored checksums differ.

Comment

Already done :
  • dCap calculates a checksum while transferring a file to the server as long as the file is writting sequentially. Any seek operation will disable the crc calculation.
  • dCap will send the calculated checksum to the server together with the close command. The server stores the checksum within a pnfs level.
  • The dCap mover can be configured to calculate a checksum when receiving data from a client. It compares the dCap and its own checksum and let the transfer fail if versions don't match.
Still to do :
Based on configuration, the pool must be able to calculate and compare checksums on 'restore' 'client fetch' as well as on manual intervention and on a frequently base.
For details :
The Pool Checksum Mechanism

Status

In Progress

Replica Manager issues

Description

Comment

Status

In Progress

Pnfs not mounted on ReadOnly pools

Description

In order to simplify configuration and management of dCache pool nodes it's desirable to avoid mounting the pnfs filesystem on those nodes.

Comment

For ReadOnly pools, the external HSM interface script needs to be changed to get rid of the pnfs mountpoint. Done for jobs/osmv2-hsmcp.sh. For ReadWrite pools, dCache code needs to be changed.

Status

Done for ReadOnly pools. Not yet scheduled for Write pools.

Simple text tables in addition to .html view (wget)

Description

In order to allow scripting of monitoring and administration tasks we will provide a text based web page counterpart for each .html web page.

Status

In progress.

Assigning unique ID to request

Description

Mainly to simplify the tracing of requests in the billing database it would be desirable to have a never repeating unique ID assigned to each request which as well is available in the HSM restore and HSM store operations.

Comment

Due to the fact that there is a many to one relation between client-dCache get and HSM get operations, we intend to only assign the ID of the first client-dcache get request to the corresponding HSM get request.

Status

To Be Done.

Author : patrick.fuhrmann@desy.de
Last Modifed : $Id: req-dcache-1.5.2.html,v 1.1 2005/04/29 12:38:58 patrick Exp $