PoolManager V5 is now available which, beside other nice features, provides pool to pool transfer if :And :
- The target file is located in a pool from which the requesting client is no allowed to read.
- The load of the pool, the target fils is located exceeds a configured value.
- PnfsId can be defined which are not handled by the PoolManager, but a defined error message is returned to the client. (Mainly to cover the case where tapes are broken and the clients continuesly try to get the tape which it is OFFLINE for repair)
- PoolManager V5 takes care that duplicated files in the cache are slowly removed as soon as the load on these file decreases.
As a first shot, we allow write pools to startup even if there are pnfs files with a status of receiving by client. They are marked BAD. These files are not regarded to be precious so they don't show up in the 'storage queues' and consequently they are not stored on the HSM. With 1.4.7B2, these files can essentially only be removed after informing the users about the loss. The pool command pf <pnfsId> might be used to get the full pathname of the dataset.The pool startup option 'recover-anyway' has to be used to have this new feature enabled. Bad files can be searched for by :
rep ls -l=e
Zero size files, and file already removed from pnfs are removed from the write storage queue as well. While zero size files are removed immediatly, 'nonexisting pnfs' files are removed as soon as they are scheduled for 'storing' to the HSM.
New pool command :
queue activate <pnfsId> | class <storageClass>@<hsm>
Optionally, pools can sent 'remove pnfs file' information to the biling cell.pool > set report remove on|off
There is a new Restore Web Page which is updated every <n> minutes. In displays the Full Pnfs Path in addition to the information provided by the already existing Restore Page.
The PnfsManager cacheinfoof command displays the statistics and the cache locations of a pnfsFile by pnfsId or Global Pnfs Path.
The PoolManager.conf can as well be a link now. On writing the setup back to the file, we are following the link and the configuration data is writting do the link destination rather then to the link itself.The PoolManager now supports to read the current configuration file back to memory overwriting the current memory setup.
PoolManager > reloadThe command first clears the 'in memory' setup and then rereads the setup file. The operation is atomic in the sense that all other PoolManager operations are suspended during the 'clear , reread' sequence.
Billing Time representation changed
The billing times have been changed from am/pm to 24 hours.Miscalculated 'removable' space on Ctrl C of write request
Any unexpected behaviour of clients writing into the cache caused the connected Pool to miscalculate the 'removable' space.Pool disables itself on Ctrl C of write request
Any unexpected behaviour of clients writing into the cache eventually caused the pool to disable itself, assuming a disk IO error.Pool to Pool transfers caused 'too many open files' error
Pool To Pool clients didn't close the file desciptors and consequently ran into 'too many open files' after a while.Heavy PnfsManager load caused 'pnfs timeouts'
Havy load of the PnfsManager manager, like pnfs un/register, caused other pnfs clients to time out. As a consequence Write requests failed due to timeouts of 'set filesize' requests.Mover doesn't react on 'mover kill <id>
Interrupting mover didn't work in most cases. It's now working, but only if there are requests from the client. In case the connection stucks, 'kill' is not honoured yet.Action Log (Billing Page) sorted now
pool : rep ls -s[=..] -> NullPointerException
Occationally the pool command rep ls -s[=..] returned with a NullPointerException.