User Tools

Site Tools


Sidebar

progetti:icarus:home

ICARUS at CNAF

Account Request

The CNAF is the main computing center of the INFN. In order to access the computing resources you need an account that can be request following the instructions provided here. An example of the form is here.

Account Password Reset

In case you lost your password, just send an email to sysop@cnaf.infn.it indicating your account and asking for password reset.

CNAF User Guide

CNAF provides a very usefull user guide for Tier1.

Monitoring

CNAF provides a very usefull user guide for Monitoring.

ICARUS UIs

In order to access any CNAF machine, you first have to login in bastion.cnaf.infn.it. From there you can login to ICARUS UIs machines:

  • ui01-icarus.cr.cnaf.infn.it
  • ui02-icarus.cr.cnaf.infn.it

The filesystems are:

  • /cvmfs: CernVM File System (CernVM-FS)
  • /storage/gpfs_archive/icarus: tape (not accessible from UIs)
  • /storage/gpfs_data/icarus/local: disk
  • /storage/gpfs_data/icarus: disk (not writable from UIs apart fileset: /storage/gpfs_data/icarus/local)
  • /opt/exp_software/icarus: dedicated to software
  • /home/ICARUS: home directories

The disk area readable from UIs are:

  • /storage/gpfs_data/icarus/plain
  • /storage/gpfs_data/icarus/plain/user/<username>

Personal Certificate and VO Enrollment

To read and write the storage areas using gfal tools first you need be enrolled in the ICARUS experiment VO: icarus-exp.org. The enrollment requires a valid personal certificate. Instructions on how to obtain it are here. Once you have a valid certificate, you have to install it in a browser and point (with that browser) to the Enrollment Url reported in operation portal site. Select the valid personal certificate just installed when asked for by the browser. Once you have filled the enrollment form, you have to wait for the VO manager to approve your enrollment.

Read/Write storage areas

Once enrolled, you need create a proxy. Instructions on how to create it are here.

The usefull tools to handle files are:

  • gfal-ls: List information about the file
  • gfal-copy: Copy file
  • gfal-rm: Remove file
  • gfal-mkdir: Create a directory

Detailed information about gfal utils can be found here

Job submission

Instructions on how to submit jobs are here. Be aware if the job needs your certificate (e.g. if the job uses gfal tools) add the following lines to the submitting script:

x509userproxy           = $(X509_USER_PROXY)
use_x509userproxy       = true

In a nutshell:

  • Launch job:
condor_submit -name sn-02.cr.cnaf.infn.it -spool run.sub
  • Monitor user jobs:
condor_q -name sn-02.cr.cnaf.infn.it <user>
  • Monitor single job:
condor_q -name sn-02.cr.cnaf.infn.it <cluster id>
  • Deeper monitoring of a job:
condor_q -better-analyze -name sn-02.cr.cnaf.infn.it <cluster id>
  • Get the output:
condor_transfer_data -name sn-02.cr.cnaf.infn.it <cluster id>

Submitting simple test production

To test everything is working correctly, clone this repository in your home and follow the istructions in "Launch the test production".

Setup icaruscode

In order to setup icarucode version vXX_YY_ZZ with qualifier eNN:PPP, just do:

source /cvmfs/icarus.opensciencegrid.org/products/icarus/setup_icarus.sh
setup icaruscode vXX_YY_ZZ -q eNN:PPP

Install icaruscode for development

wd="/opt/exp_software/icarus/users/${USER}/my_dev_area"
vers="v09_37_02_06"
qual="e20:prof"
mkdir -p ${wd}
cd ${wd}
source /cvmfs/icarus.opensciencegrid.org/products/icarus/setup_icarus.sh
setup icaruscode ${vers} -q ${qual}
export MRB_PROJECT=icaruscode
mrb n -v ${vers} -q ${qual}
source ${wd}/localProducts_icaruscode_${vers}_${qual/:/_}/setup
cd srcs
mrb g -t ${vers} icaruscode
cd $MRB_BUILDDIR
mrbsetenv
mrb install -j 2

Samweb (workaround)

It is possible to use samweb at CNAF (with a workaround) using the x509 proxy created in icarusgpvm* machines. As prerequisite, you need a valid x509 proxy in icarusgpvm* machines. In order to create it, once you are logged in at icarusgpvm* machine, do:

setup cigetcert
cigetcert -s 'fifebatch.fnal.gov'
voms-proxy-init -noregen -rfc -voms 'fermilab:/fermilab/icarus/Role=Analysis'

Then, in the cnaf machine do:

fnal_user="mtenti"
kinit -A -r7d -l26h -f ${fnal_user}@FNAL.GOV
scp ${fnal_user}@icarusgpvm01.fnal.gov:/tmp/x509up_u$(ssh ${fnal_user}@icarusgpvm01.fnal.gov 'id -u') /tmp/x509up_u$(id -u)
source /cvmfs/icarus.opensciencegrid.org/products/icarus/setup_icarus.sh
setup fife_utils

You can also add a function to your .profile

function setup_fife_utils() {
  fnal_user="mtenti"
  klist -s;
  ret=$?;
  if [ "${ret}" != "0" ]; then
    kinit -A -r7d -l26h -f ${fnal_user}@FNAL.GOV;
  fi;
  scp ${fnal_user}@icarusgpvm01.fnal.gov:/tmp/x509up_u$(ssh ${fnal_user}@icarusgpvm01.fnal.gov 'id -u') /tmp/x509up_u$(id -u)
  setup fife_utils;
}

In the following links to webpage with usefull info:

Locate a file

To locate a file which filename is filename.root, do:

filename="filename.root"
md5=$(echo -n "user.icaruspro:${filename}" | md5sum)
ls -lrt /storage/gpfs_data/icarus/plain/user/icaruspro/${md5:0:2}/${md5:2:2}/${filename}  

Example: how to get the list of the file paths for a samweb selection

  • First, you connect to icarusgpvm* machines and generate a valid proxy with the commands:
source /cvmfs/icarus.opensciencegrid.org/products/icarus/setup_icarus.sh;
setup cigetcert
cigetcert -s 'fifebatch.fnal.gov'
voms-proxy-init -noregen -rfc -voms 'fermilab:/fermilab/icarus/Role=Analysis'
  • Then, you copy the proxy to CNAF machines:
fnal_user="mtenti"
kinit -A -r7d -l26h -f ${fnal_user}@FNAL.GOV
scp ${fnal_user}@icarusgpvm01.fnal.gov:/tmp/x509up_u$(ssh ${fnal_user}@icarusgpvm01.fnal.gov 'id -u') /tmp/x509up_u$(id -u)
  • Then, you setup samweb:
source /cvmfs/icarus.opensciencegrid.org/products/icarus/setup_icarus.sh
setup fife_utils
  • At this point you are able to query samweb:
samweb -e icarus list-files "run_number = 8460 and data_stream = bnbmajority and data_tier = raw" > list_filenames.txt
  • Finally, you have to convert the filename into filepath using the following commands:
for filename in $(cat list_filenames.txt); 
do 
  md5=$(echo -n "user.icaruspro:${filename}" | md5sum); 
  filepath="/storage/gpfs_data/icarus/plain/user/icaruspro/${md5:0:2}/${md5:2:2}/${filename}"; 
  if [ -f ${filepath} ]; 
  then 
    echo ${filepath} >> list_filepaths.txt; 
  else 
    echo ${filename} >> list_missing_files.txt; 
  fi;
done;
  • You will have:
    • the list of the path of the existing files in list_filepaths.txt
    • the list of the name of the missing files in list_missing_files.txt

FNAL -> CNAF file copy

To be able to copy files from FNAL to CNAF you need (only once):

  • to be enrolled in the VO. See session above.
  • to create vomses folder in your home in the FNAL machines, using the following command on the FNAL machines:
mkdir -p ~/vomses
  • copy the content of /etc/vomses/icarus-exp.org in the CNAF machines in the created folder, using the following command on the CNAF machines:
scp -r /etc/vomses/icarus-exp.org/* "fnal_user"@icarusgpvm01.fnal.gov:~/vomses/.

Then, you do, ignoring possible errors:

voms-proxy-init --voms icarus-exp.org --valid 72:00 -vomses ~/vomses/icarus-exp.org-vomsigi-na.unina.it
gfal-ls srm://storm-fe-archive.cr.cnaf.infn.it:8444/srm/managerv2?SFN=/<space token>

where <space token> can be:

  • icarus for tape (i.e. /storage/gpfs_archive/icarus)
  • icarusplain for plain area on disk (i.e. /storage/gpfs_data/icarus/plain)
  • icarusdata for not plain area on disk (i.e. /storage/gpfs_data/icarus)

TITUS Event Display

source /opt/exp_software/icarus/titus/titus_v09_37_02_06/setup.sh
evd.py -i <file.root>

Support

For any kind of support refer to user-support@cnaf.infn.it

progetti/icarus/home.txt · Last modified: 2023/01/12 11:43 by tenti@infn.it