Common HEP UNIX User Environment - Towards Unification


Some time ago I sent a mail about SITE and GROUP_DIR variables. I have been delayed many times since then but eventually here is a proposal about several topics:

  1. group hook at HEP level scripts
  2. better agreement on what should be the HEP level variables, what should be their default. We should probably provide HEP level sys.conf.[c]sh files.
  3. agreement on naming schemes for some HEP variables
    1. "HEP_SITE" variable
    2. "OS" variable(s)
  4. "replication" of group and site level scripts accross HEP sites

  1. Group Hook at HEP level scripts

    The current situation is the following. As I said already in my first mail when we reviewed the HEPiX with Axel in spring 1994, we had no strong recommendations for a group level variable and we didn't understand that the hook should be put at HEP level.

    This is unacceptable anymore because collaborations are not Site dependent: they are spread over the world!!! We must do something!

    Moreover, Thomas's remark is very good when he mentioned that: "There are still a lot of people who do not believe in this support and which are still copying bad dot files from one account to another. I believe this support is rather essential as the hepix scripts itself because this is the support for group independent site hopping".

    And I believe that Thomas's point of view is the good one.

    Now the situation is the following:

    The HEP level has builtin hooks to provide a Site level customisation. The Site level is RESPONSIBLE for providing or not a Group level the way the Site managers want it.

    So, we have:

      	HEP     calling
              |----------------> Site     calling
    				|---------------> System     -|
     				|---------------> Cluster     |-customisations
    				|---------------> Group      -|
    	
     
    My proposal is that we move to a situation where

    The HEP level has a builtin hook to provide a Site level customisation but is also responsible for providing at least a Group level and should may be provide all the other hooks,

    So we would have

      	HEP     calling
              |----------------> Site       -|
    	  |----------------> System      |
     	  |----------------> Cluster     |-customisations
    	  |----------------> Group      -|
    
    
    Then, we would have a common way to access the "group" customisations. Keep in mind that as the system retrieves its information across the information provided by the previous levels, site, system, cluster, it is always possible that any site implements its own location for group customisation files.

    This implies that we must agree on the name of environment variable which is going to store the location of the group files.

    1. the HEPiX scripts evaluate which is the group path from the HEP, Site System and Cluster levels.
    2. the value for the group hook is then set. The HEPiX scripts can retrieve any group level customisation.

    My proposal would be to use HEP_GROUP_DIR variable to store the location of the group level customisation.

  2. Better agreement on what should be the HEP level variables

    Some variables are said to be HEP variables, like OS, USERPATH, etc. (The variable SITE could be another candidate here). Although these variable must be set by the site administrator, it doesn't mean that they have a meaning at HEP level.

    But, the OS variable is a good example of the kind of problems we are facing:

    We agreed that we need this variable but we never agreed on the value.

    So, I propose that we review these variables and check whether they should be set at Site level or at HEP level.

    If they require to be set at HEP level, then I would propose that there exists an HEP_sys.conf.[c]sh file which would be sourced by the HEP_csh.cshrc or HEP_zshenv files.

  3. Agreement on naming schemes for some HEP variables

    1. "HEP_SITE" variable

      It could be useful to have an HEP_SITE variable. Although people should tell me what is the aim of this variable. It could be like at DESY used for hostname to be not fully qualified? Is it something we want? What does IN2P3 want to do with it? Do you want to access customisation files which maybe Site dependent because you are spread over France? Do you want to identify sub-sites? Sould we use the "IP" naming scheme like in my previous mail 'desy.de' or a HEP invented naming scheme like 'DESY', 'CERN', 'IN2P3', etc.

    2. "OS" variable(s)

      The variable OS is a very nasty example because, what should be its value: The output of for example a 'uname' command? How then to distinguish between SunOS and Solaris? Do we want to distinguish between HP-UX9 and HP-UX10? AIX3 and AIX4?

      A proposal would be that:

      1. the value of OS is the output from the uname command unless we have to distinguish (example is SunOS, Solaris). So we would have OS=HP-UX regardless of the version.
      2. Arnaud Taddei initially proposed that we introduce the variable HEP_ADV_OS with a value which is more detailed. The naming scheme could be:
        	HEP_ADV_OS = $OS$version
        
        and then we would end with:
                       HEP_ADV_OS=AIX4
                       HEP_ADV_OS=HP-UX10
         	       ...
        
      3. A good and fast OS naming-scheme conversion tool. An example could be based on Thomas's bintype command. We could as well use the GNU's naming scheme. Any proposal here would be welcome.

      Another variant for this proposal would be (from Lionel Cons):

      "I think we need at least a "simple" variable (OS = HP-UX) and a more complete information either with one "complete" variable (ADV_OS = HP-UX_9.01) or with other separate variables like (OS_VERS = 9.01).

      I don't think we should invent new names like Solaris, BTW SunOS 4 is also Solaris 1... I vote for using uname output.

      Why not:

      	 if ($OS =  "SunOS" && $OS_VERS >= 5) then
               	... solaris 2 code
      	 endif
      
      If you really want to use other names, you can also consider GNU's naming scheme."

  4. Replication of group and site level scripts accross HEP sites

    Because the group settings are cross boundaries (Atlas, CMS, etc.) somebody working in Atlas at RAL should have the same group setting as somebody working in Atlas at CERN.

    AFS allows us to access each others settings, but I believe that speed is not acceptable. Thus, my feeling, we need a "replication" mechanism between the sites and a way to exchange hepix group settings templates.

    Now, my ideas in this area are very fuzzy in respect of the organisation of such a mechanism. Any feedback is welcome.

  5. Annex: my previous mail
    From taddei@dxcern.cern.chThu Oct 12 10:31:59 1995
    Date: Wed, 7 Jun 1995 09:28:01 +0200 (METDST)
    From: Arnaud Taddei 
    To: Wojcieh Wojcik 
    Cc: Thomas Finnern , Axel Koehler ,
        Lionel Cons 
    Subject: SITE and GROUP_DIR variables
    
    
    Hi Wojcieh,
    
    	I hope you had a good trip back to France. I was pleased
    to meet you and discuss with you
    
    
    SITE variable
    -------------
    on the SITE variable, just to show you how it starts:
    
    			Variable	Default
    	----------------------------------------------
    	CERN:  		no such variable
    	DESY Hamburg:	SITE		desy.de
    	DESY Zeuthen:	no such variable
    	IN2P3:		SITE		CCIN2P3
    
    Question: should we set this variable to:
    
    	1 - a HEP name like CCIN2P3
    	2 - an 'afs' or 'dfs' kind of name like 'desy.de', 'cern.ch', etc.
    
    In the 1st case the advantage is that you don't rely on the presence of
    AFS or DFS you are not bound with it.
    
    In the 2nd case the advantage is that you can automatically calculate
    the SITE variable and if you bet AFS and DFS cells are going to grow
    in the future, then it's OK.
    
    
    This is just a small discussion to collect ideas. 
    Nobody should put too much 'religion' into it.
    	
    
    GROUP_DIR variable
    ------------------
    As far as I remember, when we designed the HEPiX scripts with
    Axel last year, we had no strong recommendations for a 
    group hook level variable. 
    
    			Variable	Default value	
    	-----------------------------------------------------------
    	CERN:   	GROUP_DIR 	/afs/cern.ch/group/$GROUP
    	DESY Hamburg:   GROUPROOT	/etc/local/groups/$GROUP
    	DESY Zeuthen:	no such variable
    	IN2P3: 		THRONG_DIR      ?
    
    This is maybe a mistake but at this period the scripts were just starting
    and we had not a single clue on the situation on the other sites
    concerning the GROUP management.
    
    So, as there is a site level in the HEPiX scripts, you can use it to
    redefine anything and build your own calls. However I believe we should
    normalise and standardise the name of the variable where are located
    the 'group level' customisation file.
    
    I wait for your input before opening a wider discussion.
    
    
    Cheers
    	
    PS: May I have a guest account at IN2P3. This will give me an
    opportunity to have a better understanding on the situation
    on your site(s).
    
    
    

Arnaud Taddei, 27-Jun-1996