Strumenti Utente

Strumenti Sito


grid:ildg

ILDG

USER GUIDE

ILDG SETUP

Startup an ILDG session:

ildg

This command execute the following steps:

source /opt/ildg/lroot/etc/env.sh

Main evironment modifications:

export LFC_HOST=grid-lfc.desy.de 
export PATH=/opt/ildg/lroot/bin:$PATH

Create the proxy:

grid-proxy-init (or voms-proxy-init -voms ildg )
ladm --show-projects

PID Collaboration Projectname   
=== ============= ===========   
 3   qcdsf         clover_nf2   
 4   gral          wilson_nf2   
 5   etmc          tmqcd_nf2    
 6   sesam         wilson_nf2   
 7   txl           wilson_nf2   
 9   dik           clover_nf2   
10   qcdsf         clover_nf2p1 
11   theta         clover_nf2   
12   etmc          tmqcd_nf2p1p1
lls --all

 Welcome to the Ltool-command  lls -

 Showing all ensembles currently in the MDC:
 ===========================================

Markov-Chain-URI                                                                              EID Read Write
================                                                                              === ==== =====
mc://ldg/dik/clover_nf2/b5p29kp13632-32x64                                                     55 All  2    
mc://ldg/dik/clover_nf2/b5p40kp13640-24x48                                                     56 All  2    
mc://ldg/dik/clover_nf2/b5p40kp13640-32x64                                                     75 All  None 
mc://ldg/dik/clover_nf2/b5p40kp13660-32x64                                                     78 All  None 
mc://ldg/etmc/tmqcd_nf2/tlSym_b3.75_L24T48_k0.1660_mu0.0200                                    53 All  6    
mc://ldg/etmc/tmqcd_nf2/tlSym_b3.8_L20T48_k0.164099_mu0.0060                                   64 All  6    
mc://ldg/etmc/tmqcd_nf2/tlSym_b3.8_L20T48_k0.164099_mu0.0090                                   65 All  6    
mc://ldg/etmc/tmqcd_nf2/tlSym_b3.8_L20T48_k0.164111_mu0.006                                    69 All  6    
mc://ldg/etmc/tmqcd_nf2/tlSym_b3.8_L24T48_k0.164111_mu0.0045                                   72 All  6    
mc://ldg/etmc/tmqcd_nf2/tlSym_b3.8_L24T48_k0.164111_mu0.006                                    71 All  6    
mc://ldg/etmc/tmqcd_nf2/tlSym_b3.8_L24T48_k0.164111_mu0.008                                    76 All  6    
...
lls mc://ldg/dik/clover_nf2/b5p40kp13640-24x48

 Welcome to the Ltool-command  lls -

 The following configurations belong to 
 mc://ldg/dik/clover_nf2/b5p40kp13640-24x48 :

lfn://ldg/dik/clover_nf2/b5p40kp13640-24x48/dik.576.00005.lime 
lfn://ldg/dik/clover_nf2/b5p40kp13640-24x48/dik.576.00010.lime 
lfn://ldg/dik/clover_nf2/b5p40kp13640-24x48/dik.576.00015.lime 
lfn://ldg/dik/clover_nf2/b5p40kp13640-24x48/dik.576.00020.lime 
...
Get Ensemble Metadata:
lget -me mc://ldg/dik/clover_nf2/b5p40kp13640-24x48

get Metadata Config
lget -m
lget -m  lfn://ldg/dik/clover_nf2/b5p40kp13640-24x48/dik.576.00005.lime

get Binary confs:
lget  --vo=ildg lfn://ldg/dik/clover_nf2/b5p40kp13640-24x48/dik.576.00005.lime

emi-ui.pr.infn.it

voms-proxy-init -voms ildg
lcg-infosites --is grid-bdii.desy.de --vo ildg  se
lfc-ls /grid/ildg
lfc-rmdir -r  /grid/ildg/parma
lfc-mkdir /grid/ildg/test/parma
lcg-del     -v -a  lfn:/grid/ildg/test/perftest.tar.gz.081125-1228
setfacl -R -m   g:ildg:rx  /storage/ildg
setfacl -R -m d:g:ildg:rx  /storage/ildg

uberftp grid-se2.pr.infn.it
cd /storage/ildg/

With File Catalog

LFC_HOST=grid-lfc.desy.de lfc-ls -l /grid/ildg/ldg
LFC_HOST=grid-lfc.desy.de lfc-ls -l /grid/ildg/ldg/dik/clover_nf2/
LFC_HOST=grid-lfc.desy.de lcg-cp  -v lfn://grid/ildg/ldg/dik/clover_nf2/b5p40kp13640-24x48/dik.576.00005.lime  file://$HOME/dik.576.00005.lime

Without LFC file catalog:

voms-proxy-init -voms ildg
lcg-lr -v --vo ildg lfn://grid/ildg/ldg/dik/clover_nf2/b5p40kp13640-24x48/dik.576.00005.lime 

srm://dcache.fz-juelich.de/pnfs/fz-juelich.de/data/ildg/qcdsf/clover_nf2/b5p40kp13640-24x48/dik.576.00005.lime
srm://dcache.zib.de/pnfs/zib.de/data/ildg/dik/clover_nf2/b5p40kp13640-24x48/dik.576.00005.lime
srm://dcache.zib.de/pnfs/zib.de/data/ildg/qcdsf/clover_nf2/b5p40kp13640-24x48/dik.576.00005.lime
srm://globe-door.ifh.de/pnfs/ifh.de/acs/grid/ildg/ldg/qcdsf/clover_nf2/b5p40kp13640-24x48/dik.576.00005.lime

voms-proxy-init -voms theophys
srmcp -2 -streams_num=1 -debug=true -webservice_path=srm/managerv2 srm://grid-se2.pr.infn.it:8444/ildg/qcdsf/clover_nf2/b5p20kp13420-16x32/ape.001.000305.dat file:////tmp/copy-from-parma

ADMIN GUIDE

ildg-ui installation (23/5/2008)

ildg-ui.pr.infn.it 192.135.11.40

Browse site: http://www-zeuthen.desy.de/ape/html/Download/ldg/

mkdir /opt/ildg
cd /opt/ildg


wget http://www-zeuthen.desy.de/ape/html/Download/ldg/setup
sh setup
LROOT=`pwd`/lroot; export LROOT

$LROOT/bin/lrpm -ivh env.rpm
$LROOT/bin/lrpm -ivh gui-4.1-1.sl54.noarch.rpm  gui-default-4.1-1.sl54.noarch.rpm  
$LROOT/bin/lrpm -ivh cert.rpm  # execute every 2 weeks with crontab

. $LROOT/etc/env.sh

lrpm -ivh jre.rpm        jre-default.rpm
lrpm -ivh ltools.rpm     ltools-default.rpm
lrpm -ivh ildg_cksum.rpm ildg_cksum-default.rpm

cat > /etc/profile.d/ildg.sh << EOF
export LROOT=/opt/ildg/lroot
. $LROOT/etc/env.sh
EOF


cat > /etc/cron.d/ildg_cert_update << EOF
1 1 * * 6 root  /opt/ildg/lroot/bin/lrpm -U cert.rpm  >& /dev/null
EOF

per un errore su lget aggiunto il link

ln -s libldap-2.2.so.7.0.6 libldap.so.2

grid-ui (29/05/2008)

/opt/glite/etc/vomses/ildg-grid-voms.desy.de

"ildg" "grid-voms.desy.de" "15111" "/C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de" "ildg"

ui_ildg.conf

 [
 VirtualOrganisation = "ildg";
 NSAddresses = "grid-rb.desy.de:7772";
 LBAddresses = "grid-rb.desy.de:9000";
 MyProxyServer = "grid-pxy.desy.de"
 ]

test-se.jdl

Type = "Job";
JobType = "Normal";
Executable = "/bin/bash";
Arguments = "test-se.bash";
InputSandbox={"test-se.bash","perftest.tar.gz"};
StdOutput = "std.out";
StdError = "std.err";
OutputSandbox = {"std.out","std.err"};

test-se.bash

#!/bin/sh -x
#data=080527-1237
data=$(date +%y%m%d-%H%M)
mydir=$(pwd)
file=perftest.tar.gz
SE=grid-se2.pr.infn.it 
VO=ildg
#VO=theophys
/opt/lcg/bin/lcg-cr   -v --vo $VO -d $SE -l lfn:/grid/$VO/test/parma/$file.$data file://$mydir/$file
/opt/lcg/bin/lcg-cp   -v --vo $VO lfn:/grid/$VO/test/parma/$file.$data  file://$HOME/$file.$data
/opt/lcg/bin/lcg-del  -v --vo $VO -a  lfn:/grid/$VO/test/parma/$file.$data 

Commands:

voms-proxy-init    -voms ildg
edg-job-submit     -o test-id1  --config-vo ui_ildg.conf   test-se.jdl
edg-job-status     -i test-id1
edg-job-get-output -i test-id1 --dir .

grid-se (28/05/2008)

my-site-info.def:

ALL_VOMS_VOS="argo     bio   infngrid cdf     compchem
              planck   enea  ingv     gridit  inaf
              theophys virgo pamela   libi    eumed
              euchina egrid  euindia  cyclops omiieurope
              compassit ildg"

VOS="infngrid theophys dteam ops ildg"

VO_ILDG_SW_DIR=$VO_SW_DIR/ildg      
VO_ILDG_DEFAULT_SE=$CLASSIC_HOST   
VO_ILDG_STORAGE_DIR=$CLASSIC_STORAGE_DIR/ildg      
VO_ILDG_VOMS_SERVERS="vomss://grid-voms.desy.de:8443/voms/ildg?/ildg"          
VO_ILDG_VOMSES="ildg grid-voms.desy.de  15111 /C=DE/O=GermanGrid/OU=DESY/CN=host/grid-voms.desy.de ildg" 

users.conf

59701:ildg001:59700:ildg:ildg::
59702:ildg002:59700:ildg:ildg::
59703:ildg003:59700:ildg:ildg::
59704:ildg004:59700:ildg:ildg::
59705:ildg005:59700:ildg:ildg::
59706:ildg006:59700:ildg:ildg::
59707:ildg007:59700:ildg:ildg::
59708:ildg008:59700:ildg:ildg::
59709:ildg009:59700:ildg:ildg::
59710:ildg010:59700:ildg:ildg::
59711:ildg011:59700:ildg:ildg::
59712:ildg012:59700:ildg:ildg::
59713:ildg013:59700:ildg:ildg::
59714:ildg014:59700:ildg:ildg::
59715:ildg015:59700:ildg:ildg::
59716:ildg016:59700:ildg:ildg::
59717:ildg017:59700:ildg:ildg::
59718:ildg018:59700:ildg:ildg::
59719:ildg019:59700:ildg:ildg::
59720:ildg020:59700:ildg:ildg::

groups.conf

#"/VO=ildg/GROUP=/ildg/ROLE=SoftwareManager":::sgm:
"/VO=ildg/GROUP=/ildg"::::

http://grid.desy.de/voms/

rpm -Uvh http://grid.desy.de/voms/lcg-vomscerts-desy-0.0.5-1.noarch.rpm  (already installed)

grid-proxy-init

L'autenticazione con grid-proxy-init non funziona (mail di Dirk del 13/12/08)

Soluzione (mail di Magnoni del 15/12/08):

Di default le storage area sono chiuse all'accesso per solo utenti voms della VO a cui la storage area appartiene e quindi funzionano solo con certificati con FQAN voms. Per aprire la storage area anche a utenti con certificati grid-proxy, editare

 /opt/storm/backend/etc/namespace.xml 

sul backend di StoRM e commentare o rimuovere la linea

<vo-name> ildg </vo-name> 

per la storage area in questione.

Per evitare la sovrascrittura di namespace.xml da parte di yaim occorre settare la variabile STORM_NAMESPACE_OVERWRITE a “false” nel file ig-se_storm_backend

29/06/2009: con la versione 1.4 di storm non c'e' piu' bisogno di questa patch

grimap-file-local

Ancora problemi con grid-proxy-init (mail di Dirk del 17/12/08)

Soluzione:

Se un utente con lo stesso certificato e' in 2 VO la seconda viene ignorata. Dobbiamo quandi aggiungere a mano le entri mancanti. Aggiungere gli utenti nel gridmapfile. (vedi /opt/edg/etc/edg-mkgridmap.conf ) Edit /opt/edg/etc/grid-mapfile-local:

"/C=IT/O=INFN/OU=Personal Certificate/L=Parma/CN=Roberto Alfieri" .ildg
"/C=IT/O=INFN/OU=Personal Certificate/L=Parma/CN=Roberto Covati" .ildg
"/C=IT/O=INFN/OU=Personal Certificate/L=Parma/CN=Francesco Di Renzo" .ildg
"/C=IT/O=INFN/OU=Personal Certificate/L=Parma/CN=Enrico Tagliavini" .ildg

srmcp

srmcp fallisce (mail di Dirk del 1/1/09)

srmcp -2 -debug=true  srm://grid-se2.pr.infn.it:8444/ildg/qcdsf/clover_nf2/b5p20kp13420-16x32/ape.001.000305.dat file:////tmp/copy-from-parma 

Soluzione di Magnoni (mail di Magnoni del 8/1/2009).

echo "extraslashes.gsiftp=/" >> /opt/storm/etc/storm.properties

Per evitare la sovrascrittura di storm.properties da parte di yaim occorre settare la variabile STORM_PROPERTIES_OVERWRITE a “false” nel file ig-se_storm_backend

/var/www/html/dokuwiki/data/pages/grid/ildg.txt · Ultima modifica: Y/m/d H:i da