Differences

This shows you the differences between two versions of the page.

--- progetti:htcondor-tf:htcondor-ce_setup [2020/09/22 16:39] – dalpra@infn.it
+++ progetti:htcondor-tf:htcondor-ce_setup [2020/11/29 12:27] (current) – dalpra@infn.it
@@ Line 1: / Line 1: @@
+==== HTCondor-CE (old) ====
+== Notes for the manual setup of a HTCondor-CE  ==
+The HTCondor-CE must be installed on a HTCondor //submit node//, that is a machine where a SCHEDD daemon runs:\\
+''[root@ce02-htc ~]# condor_config_val DAEMON_LIST\\
+MASTER, SCHEDD, COLLECTOR''
+Furthermore:
+   * It must also hold a valid X509 host certificate.
+   * Must be reachable from everywhere to the TCP port 9619.
+   * The repo for the latest stable release (3.2.1 as of writing) is:
+     * [[https://research.cs.wisc.edu/htcondor/yum/repo.d/htcondor-stable-rhel7.repo]]
+   * Needed RPMs are:
+     *  htcondor-ce-bdii, htcondor-ce-client, htcondor-ce-view, htcondor-ce, htcondor-ce-condor
+   * To finalize the configuration one may refer to the official documentation here: [[https://htcondor-ce.readthedocs.io/en/stable/|HTCondor-CE]].
+   * A few supplementary notes for non OSG people are provided below
+== VOMS e Grid ==
+Official documentation points to [[https://opensciencegrid.org/docs/security/lcmaps-voms-authentication/|lcmaps-voms-authentication]] which refers to OSG. We could instead refer to the standard configuration of a CREAM-CE, which only needs minor changes:\\
+\\
+''ln -s /etc/grid-security/voms-grid-mapfile /etc/grid-security/voms-mapfile\\
+cp -a /etc/grid-security/hostcert.pem /etc/grid-security/condorcert.pem\\
+cp -a /etc/grid-security/hostkey.pem /etc/grid-security/condorkey.pem\\
+chown condor.condor /etc/grid-security/condorcert.pem /etc/grid-security/condorkey.pem\\
+''
+== GSI and authz ==
+HTCondor-CE relies on argus for authorization. A basic configuration example for argus his detailed below.
+''
+[root@ce02-htc grid-security]# cat /etc/grid-security/gsi-authz.conf\\
+globus_mapping /usr/lib64/libgsi_pep_callout.so argus_pep_callout\\
+[root@ce02-htc grid-security]#\\
+[root@ce02-htc grid-security]# cat /etc/grid-security/gsi-pep-callout-condor.conf\\
+pep_ssl_server_capath /etc/grid-security/certificates/\\
+pep_ssl_client_cert /etc/grid-security/condorcert.pem\\
+pep_ssl_client_key /etc/grid-security/condorkey.pem\\
+pep_url https://<tuo.argus.server.preferito>:8154/authz\\
+pep_timeout 30 # seconds\\
+xacml_resourceid http://<tuo.argus.server.preferito>/condor-ce\\
+\\
+''
+== GSI and argus ==
+Refer to the official documentation: [[https://argus-documentation.readthedocs.io/en/stable/|ARGUS]] for general help about installation and configuration. We provide below an example set of configurations for the HTCondor-CE:
+''
+[root@argus ~]# pap-admin lp\\
+\\
+default (local):\\
+\\
+resource "http://your.favourite.argus.server/condor-ce" {\\
+    obligation "http://glite.org/xacml/obligation/local-environment-map" {\\
+    }\\
+\\
+    action ".*" {\\
+        rule permit { vo="ops" }\\
+        rule permit { vo="dteam" }\\
+        rule permit { vo="alice" }\\
+        rule permit { vo="atlas" }\\
+        rule permit { vo="ams02.cern.ch" }\\
+        rule permit { vo="xenon.biggrid.nl" }\\
+    }\\
+}\\
+''
+  * You can consider installing the argus service on the HTC-CE host itself. This would probably ease early setup.
+  * Be aware that Argus need read/write access to the ''/etc/grid-security/gridmapdir'' directory, which is usually hosted on a shared filesystem.
+To verify that your Argus service is properly configured to work with your HTC-CE:
+  * Create a valid proxy of a supported VO (i.e.: ''voms-proxy-init --voms cms'')
+  * install an argus pep client on your HTC-CE (''yum install argus-pepcli'')
+  * copy the proxy on the root dir of your HTC-CE as ''user509.pem''
+  * Execute the following example command (adapt to your case)
+''
+[root@htc-ce-02 ~]# pepcli --pepd https://your.argus.server:8154/authz --keyinfo user509.pem --capath /etc/grid-security/certificates --cert /etc/grid-security/hostcert.pem --key /etc/grid-security/hostkey.pem --resourceid "http://your.argus.server/condor-ce" -a pippo
+''
+On a working setup you should see an output like:
+''
+Resource: http://your.argus.server/condor-ce\\
+Decision: Permit\\
+Obligation: http://glite.org/xacml/obligation/local-environment-map/posix (caller should resolve POSIX account mapping)\\
+Username: cms195\\
+Group: cms\\
+Secondary Groups: cms\\
+''
+As a further check you should see that the empty file
+''/etc/grid-security/gridmapdir/cms195'' has a static link whose name is derived from the User DN Subject, which can be identified this way:
+''\\
+[root@argus ~]# ls -li /etc/grid-security/gridmapdir/cms195\\
+-rw-r--r-- 2 root root 0 26 giu  2019 /etc/grid-security/gridmapdir/cms195\\
+[root@argus ~]# ls -li /etc/grid-security/gridmapdir/ | grep 383663\\
+-rw-r--r-- 2 root root 0 26 giu  2019 %2fdc%3dch%2fdc%3dcern%2fou%3dorganic%20units%2fou%3dusers%2fcn%3dsurname%2fcn%3d606831%2fcn%3dname%20surname:cms:cmsprd:cms
+-rw-r--r-- 2 root root 0 26 giu  2019 cms195\\
+''
+== condor_mapfile ==
+An entry has to be ADDED to the condor_mapfile to match the certificate DN for the hosts at your site.\\
+These are mapped to the value defined by UID_DOMAIN in HTCondor and HTCondor-CE (in our case: "htc_tier1")
+''\\
+[root@ce02-htc ~]# cat /etc/condor-ce/condor_mapfile\\
+GSI "^\/DC\=com\/DC\=DigiCert-Grid\/O=Open Science Grid\/OU\=Services\/CN\=(host\/)?([A-Za-z0-9.\-]*)$"\2@daemon.opensciencegrid.org\\
+GSI "^\/DC\=DigiCert-Grid\/DC\=com\/O=Open Science Grid\/OU\=Services\/CN\=(host\/)?([A-Za-z0-9.\-]*)$" \2@daemon.opensciencegrid.org\\
+GSI "^\/DC\=org\/DC\=opensciencegrid\/O=Open Science Grid\/OU\=Services\/CN\=(host\/)?([A-Za-z0-9.\-]*)$" \2@daemon.opensciencegrid.org\\
+GSI "^\/DC=ch\/DC=cern\/OU=computers\/CN=?([A-Za-z0-9.\-]*)$" \1@cern.ch\\
+GSI "^\/C\=IT\/O\=INFN\/OU\=Host\/L=CNAF\/CN\=(host\/)?([A-Za-z0-9.\-]*)$" \1@htc_tier1\\
+GSI (.*) GSS_ASSIST_GRIDMAP\\
+GSI "(/CN=[-.A-Za-z0-9/= ]+)" \1@unmapped.opensciencegrid.org\\
+CLAIMTOBE .* anonymous@claimtobe\\
+FS (.*) \1\\
+\\
+''
+=== Configuration ===
+The default configuration path is ''/etc/condor-ce/config.d/''.
+A tool ''condor_ce_config_val'' is provided to inspect configurations, in the same way as ''condor_config_val''.
+== Examples ==
+  * Find //where// and //how// is defined an identifier (Knob) whose name is exactly known:
+ ''[root@ce03-htc ~]# condor_ce_config_val -v HTCONDORCE_VONames\\
+HTCONDORCE_VONames = alice, atlas, cdf, cms, dteam, lhcb, virgo\\
+ # at: /etc/condor-ce/config.d/06-ce-bdii.conf, line 14''
+  * See names and value for identifiers matching a part:
+''[root@ce03-htc ~]# condor_ce_config_val -d HTCONDORCE\\
+# Configuration from machine: ce03-htc.cr.cnaf.infn.it\\
+\\
+# Parameters with names that match HTCONDORCE:\\
+HTCONDORCE_BDII_ELECTION = LEADER\\
+HTCONDORCE_BDII_LEADER = ce03-htc.cr.cnaf.infn.it\\
+HTCONDORCE_CORES = 16 # cores per node\\
+[...]
+''
+Most of the predefined values already have reasonable values and there should be no reason to alter them; by individually inspecting  the configuration files there are comments providing hints about what should be customized.
+Worth to mention entries are :
+  * ''HTCONDORCE_VONames = alice, atlas, cdf, cms, dteam, lhcb, virgo''
+  * ''DAEMON_LIST = $(DAEMON_LIST), CEVIEW, GANGLIAD''\\
+from file ''/etc/condor-ce/config.d/05-ce-view.conf''\\
+This enables a CE monitoring webtool (CEView) which is visible as http://your.ce.fully.qualified.name:80\\
+Atr this point the CE should be able to handle firts job submissions
+=== Testing the CE ===
+  * From a User Interface having the ''htcondor-ce-client'' rpm, after generating a valid proxy, the CE can be tested by \\
+''condor_ce_trace <fqdn del CE>''\\
+use the ''--debug'' option to get more details.
+=== BDII ===
+the rpm creates two configuration files and python script:
+''\\
+[root@ce02-htc bdii]# rpm -ql htcondor-ce-bdii\\
+/etc/condor/config.d/50-ce-bdii-defaults.conf\\
+/etc/condor/config.d/99-ce-bdii.conf\\
+/var/lib/bdii/gip/provider/htcondor-ce-provider\\
+''
+**Note1:** the path is under /etc/condor/, thus these are HTCondor configurations, not HTCondor-CE;
+**Note2:** i manually had to define one more knob:
+''GLUE2DomainID = $(HTCONDORCE_SiteName)''
+Otherwise the script ''htcondor-ce-provider'' would exit with error. (update: fixed in more recent versions)
+To check that the configuration is formally fine just execute /var/lib/bdii/gip/provider/htcondor-ce-provider;
+a dump of the glue2 schema shoud appear on ''stdout''.
+finally, activate the service with
+''systemctl enable bdii\\
+systemctl start bdii''