Table of Contents
Abstract
This document contains the information concerning the libraries provided by the module gLite Data Agent Common.
The purpose of that package is to collect together libraries that could be used bt the gLite Data Transger Agents and the gLite Data Scheduler Agents. These libraries are:
Provides some utility functions for parsing URLs and SURLs, setting the X509_USER_PROXY environmen variable and defines the base classes of the exception hierarchy.
Provides a wrapper on top of the gLite ServiceDiscovery library, as well as a Cache, in order to reduce the number of call to the underlying InfoSys, and some utility functions for the common cases.
Define the interface (using the AbstractFactory pattern) for interacting with an Credential Service and provides the implementation of this interface for contacting MyProxy.
Define the implementation of some base classes for accesing a Data Source using the DAO (Data Access Object) Pattern. Supported backend are Oracle and MySql.
Define the some helper classes for embedding the Python interpreter inside the Agent process, in order to allow to improve the extensibility using some python scripts. It also provide the python modules that wrap the SD, Cred and Common utilities, provided by the above packages.
The gLite Data Agents Common modules use the gLite Data Config Service module for retrieving initialization and configuration parameters. For a detailed explanation how that library works please have a look at the gLite Data Config Service User's Guide.
The purpose of this section is to illustrate the initialization and configuration parameters required by the gLite Agents Common modules.
Represents the module that should be use to retieve service information from ServiceDiscovery
libglite_data_agents_common_sd.so
The type used to register SRM Services on ServiceDiscovery
Type: string; Default value: 'SRM'
The type used to register GRIDFTP Servers on ServiceDiscovery
Type: string; Default value: 'gsiftp'
The type used to register MyProxy Catalog Services on ServiceDiscovery
Type: string; Default value: 'MyProxy'
The type used to register FileTransfer Services on ServiceDiscovery
Type: string; Default value: 'org.glite.FileTransfer'
The type used to register FileTransferStat Services on ServiceDiscovery
Type: string; Default value: 'org.glite.FileTransferStats'
The type used to register ChannelManagement Services on ServiceDiscovery
Type: string; Default value: 'org.glite.ChannelManagement'
The type used to register Channel Agents on ServiceDiscovery
Type: string; Default value: 'org.glite.ChannelAgent'
The type used to register BDII Services on ServiceDiscovery
Type: string; Default value: 'BDII'
The type used to register VOMS Services on ServiceDiscovery
Type: string; Default value: 'org.glite.voms'
The type used to register SEIndex Services on ServiceDiscovery
Type: string; Default value: 'org.glite.SEIndex'
The type used to register Fireman Services on ServiceDiscovery
Type: string; Default value: 'org.glite.Fireman'
The name of the SRM Service Property that provides the path to be used. This is computerd starting from the Glue SAPath property
Type: string; Default value: 'SEMountPoint'
The name of the ChannelAgent Service Property that provides the channel source site.
Type: string; Default value: 'source'
The name of the ChannelAgent Service Property that provides the channel destination site.
Type: string; Default value: 'destination'
The name of the MyProxy Server Property that provides the Fts Mode.
Type: string; Default value: 'FtsMode'
The value of the MyProxy Server FtsMode Property for retrivers.
Type: string; Default value: 'retriever'
The value of the MyProxy Server FtsMode Property for renewers.
Type: string; Default value: 'renewer'
Enable caching of ServiceDiscovery information
Type: boolean; Default value: 'true'
The validity, in seconds, of the entries created in the cache. When this time elapsed, the entry is refreshed automatically by the cache next time the user asks for it. Entry related to queries that returned no result are not refreshed. If this parameter is not provided, the internal default will apply (1 hour)
Type: integer; Default value: ''
The reduced validity time for the stale entries in the cache: When the validity of an entry expires, the cache try to refresh it using ServiceDiscovery. In case of failure, the entry is considered stale and kept into the cache with a shorter validity. If this parameter is not provided, the internal default will apply (15 minutes)
Type: integer; Default value: ''
The number of seconds that should elaps before the entry in the cache are considered obsolete and can then be purged. Since entries are periodically refreshed based on the usage, the purge operation affects only stale entries. If this parameter is not provided, the internal default will apply (5 hours)
Type: integer; Default value: ''
The number of seconds that should elaps before considering obsolete the entries related to queries that didn't return any result. These "negative" results are not automatically refreshed, therefore should be cleaned more often. If this parameter is not provided, the internal default will apply (5 minutes)
Type: integer; Default value: ''
The gLite Agent Cred module support the Connectors/Plug-ins architecture via the AbstractFactory Pattern, therefore it would be possible to use different services by simply loading at run-time the correct connector. This can be achieved using the gLite Data Config Service module by inserting the related component inside the configuration file. At the moment the only implementation available is MyProxy:
Represents the mmodule that provide a client to the MyProxy Server for retrieving proxy certificates to be used for the File Transfers
libglite_data_agents_common_cred_myproxy.so
The host name of the MyProxy Server. If that parameter is not set or is empty, the Myproxy Server is looked up using the Service Discovery and then, if not found, the myproxy default will apply (MYPROXY_SERVER environment variable)
Type: string; Default value: ''
The port of the MyProxy Server. If that parameter is not set or is 0, the myproxy default will applies
Type: integer; Default value: '0'
The lifetime in seconds of the proxy certificates that will be created
Type: integer; Default value: '86400'
The location where the certificates retrieved from the MyProxy Service will be stored. That location must already exist
Type: string; Default value: '/tmp'
The minimum validity time (in seconds) an already existent certificate should have before submitting a new job. In case the certificate couldn't satisfy that requirement, a new certificate will be retrieved from the MyProxy Service
Type: integer; Default value: '3600'
<No configuration parameters>
The goal of the gLite Agent DAO module is to provide, based on the AbstractFactory Pattern, a common way to configure and setup connections to a DataSource in order then to implement a Agent specific DAO layer. At the moment two backends are supported: MySQL and Oracle
Represents the base module to Access Data (DAO) using Oracle as Data Source
libglite_data_agents_common_dao_oracle.so
The Oracle ConnectString identifying the DB
Type: string; Default value: 'localhost'
the name of the user that should be used to connect to the DB
Type: string; Default value: ''
the password of the user that should be used to connect to the DB
Type: string; Default value: ''
The Size of the statement Cache.0 means that the caching is disabled. Note: since some memory leaks has been observed, it better for the moment to keep the cache disabled
Type: integer; Default value: '0'
<No configuration parameters>
Represents the basic module to Access Data (DAO) using MySQL as Data Source
libglite_data_agents_common_dao_mysql.so
The name of the DB Server where mySQL is installed
Type: string; Default value: 'localhost'
The port of the mySQL DB Server. 0 means the mySQL default
Type: integer; Default value: '0'
the name of the Database holding the queue
Type: string; Default value: 'transfer'
the name of the user that should be used to connect to the DB
Type: string; Default value: ''
the password of the user that should be used to connect to the DB
Type: string; Default value: ''
The name of the socket to be used to connect to the DB Server in case <HostName> is "localhost". If no value is specified, the mysql default is used
Type: string; Default value: ''
<No configuration parameters>
The purpose of this module is to load and initialize the Python interpreter in order to run python script inside an Agent process. The module also initialize some wrapper modules that expose the main agents-common functionalities
libglite_data_agents_common_python.so
The location where the python interpret look for modules and packages. The value can be a list of directory path separated by ':' (in a unix-like sistem). If this property is set, the module would add the value to the PYTHONPATH environment variable
Type: string; Default value: ''
<No configuration parameters>
The folder config contains the configuration templates for each component provided by the gLite Agent Common module. These files are then exported in $(GLITE_LOCATION)/share/config/glite-data-agents-common/config-templates. The gLite Data Transfer and Scheduler Agents would the use these component configuration template files and combine them in order to provide the Agent's service configuration template that can be used to generate the configuration file as explained in the gLite Data Config Service Users' Guide and in the Agent related Install Guide documents.
All the gLite Data Agents Common Python component provides some utility function to embed a python interpreter inside the Agent process and exports as well some wrappers on the utility libraries provided by this module. In order to do that, you should include the agents-python component section in the service configuration file, as explained in the gLite Data Config Service Users' Guide.
Please not that the version of the Python interpreter supported by this module depends on the platform where the module is built, therefore for compatibility reason, it's that the scripts that are invoked via this mechanism assume that as reference version Python 2.2