Differences

This shows you the differences between two versions of the page.

--- progetti:icarus:tutorial [2024/02/19 22:15] – tenti@infn.it
+++ progetti:icarus:tutorial [2024/02/21 09:32] (current) – tenti@infn.it
@@ Line 1: / Line 1: @@
+====== TUTORIAL ======
+==== Video ====
+Here the link to the [[https://pandora.infn.it/public/48358e|video]]
+==== Prerequisites ====
+To run this tutorial you need:
+  - access to the ICARUS machines at CNAF. Instructions on how to request it can be found [[https://wiki.infn.it/progetti/icarus/home#account_request|here]].
+  - a valid x509 personal certificate and an enrollment to ICARUS vo. Instructions [[https://wiki.infn.it/progetti/icarus/data#personal_certificate_and_vo_enrollment|here]]
+==== Connect to ICARUS ui ====
+First you have to connect to ''bastion.cnaf.infn.it'':
+  ssh -X <user>@bastion.cnaf.infn.it
+Then, you could connect to ''ui01-icarus.cr.cnaf.infn.it'' or ''ui02-icarus.cr.cnaf.infn.it''
+  ssh -X <user>@ui01-icarus.cr.cnaf.infn.it
+==== Certificate Installation on a user interface ====
+You have to install the x509 certificate into the user interface. To do so, first copy the certificate into the user interface with the command:
+  scp certs.p12 <user>@bastion.cnaf.infn.it:~
+and from there again,
+  scp certs.p12 <user>@ui02-icarus.cr.cnaf.infn.it:~
+and then follow the instructions [[https://confluence.infn.it/display/TD/8+-+Digital+Personal+Certificates+and+Proxies+management|here]] in the section **Certificate Installation on a user interface**.
+==== Setup the ICARUS environment ====
+Run the following script to setup the ICARUS environment:
+  source /cvmfs/icarus.opensciencegrid.org/products/icarus/setup_icarus.sh
+==== Create proxy woth voms extensions ====
+Create the proxy with the voms extensions:
+  voms-proxy-init --voms icarus-exp.org --valid 72:00
+**N.B. Be sure to have proxy with a suitable validity, everytime you submit a production job.**
+To check the validity of the proxy, run the command:
+  voms-proxy-info --all
+==== Create your working area ====
+Now you can create and go to your working area. For example
+  mkdir -p /storage/gpfs_data/icarus/local/users/tentiicarus/tutorial_20240219
+  cd /storage/gpfs_data/icarus/local/users/tentiicarus/tutorial_20240219
+==== Clone the repository ====
+Now, you can clone the repository with the machinery for the production. A dedicated branch has been created **just** for this tutorial.
+  git clone -b tutorial_20240219 https://baltig.infn.it/icarus/prod-scripts.git
+==== Customize the configuration  ====
+Now, you can have a look at the file //variable.sh//. A detailed description of its content can be found [[https://baltig.infn.it/icarus/prod-scripts|here]].
+  cd prod-scripts
+  vim variables.sh
+Change the variable YOUR_CUSTOM_RUN_LIST. For example
+  YOUR_CUSTOM_RUN_LIST="9331"
+The list of the RUN2 runs is:
+  9308  9327  9344  9366  9415  9515  9569  9596  9692  9726  9749  9764  9794  9838  9862  9895  9945  9967
+  9309  9328  9345  9367  9433  9522  9570  9597  9693  9727  9750  9765  9796  9839  9870  9896  9949  9968
+  9310  9329  9346  9380  9435  9531  9571  9599  9694  9728  9751  9766  9798  9840  9872  9897  9950  9969
+  9311  9330  9347  9383  9436  9532  9580  9602  9695  9729  9752  9767  9799  9841  9873  9913  9951  9970
+  9312  9331  9354  9384  9437  9533  9582  9610  9696  9730  9753  9768  9800  9842  9874  9914  9952  9971
+  9314  9332  9355  9385  9438  9534  9583  9624  9699  9731  9754  9769  9801  9843  9875  9916  9953  9972
+  9318  9333  9356  9386  9439  9558  9587  9642  9700  9732  9755  9781  9802  9844  9876  9917  9954  9974
+  9319  9335  9357  9387  9441  9559  9588  9646  9704  9733  9756  9782  9804  9847  9886  9919  9956  9976
+  9320  9336  9358  9388  9442  9560  9589  9648  9715  9735  9757  9783  9805  9849  9887  9921  9959  9977
+  9321  9337  9359  9389  9444  9562  9590  9649  9716  9743  9758  9784  9806  9851  9888  9922  9960  9979
+  9322  9338  9360  9390  9445  9563  9591  9673  9717  9744  9759  9785  9807  9852  9889  9924  9961  9981
+  9323  9339  9361  9391  9448  9564  9592  9674  9720  9745  9760  9786  9814  9853  9890  9925  9963  9982
+  9324  9340  9363  9392  9451  9565  9593  9688  9721  9746  9761  9788  9834  9854  9891  9926  9964  9983
+  9325  9342  9364  9409  9513  9566  9594  9690  9723  9747  9762  9791  9835  9855  9892  9941  9965  9984
+  9326  9343  9365  9412  9514  9568  9595  9691  9725  9748  9763  9792  9837  9860  9894  9944  9966  9986
+==== Submit the jobs  ====
+Submit the jobs running the script //submit_production.sh//
+  ./submit_production.sh
+==== Monitor the jobs  ====
+You can check the jobs with the command:
+  condor_q -name sn-02.cr.cnaf.infn.it
+In case something was wrong, you can debug the submission retrieving the log, err and out files with the command:
+  condor_transfer_data -name sn-02.cr.cnaf.infn.it <job id>
+After a while the results of your submission should appear in ///storage/gpfs_data/icarus/plain/data/tutorial_20240219//.
+==== Resubmission ====
+If some job fails, you can resubmit the the job with the command:
+  ./resubmit <RUN-NUMBER> <STREAMS>
+For example,
+  ./resubmit 9331 "BNBMAJORITY BNBMINBIAS"
+==== CNAF monitoring webpage ====
+CNAF provides a [[https://t1metria.cr.cnaf.infn.it/d/DPv3p6zGz/batch-overview?orgId=18&refresh=5m&from=now-7d&to=now&var-retention=one_week&var-queue=icarus|webpage]] to monitor the ICARUS job queue.