==== HTCondor-CE (old) ==== == Notes for the manual setup of a HTCondor-CE == The HTCondor-CE must be installed on a HTCondor //submit node//, that is a machine where a SCHEDD daemon runs:\\ ''[root@ce02-htc ~]# condor_config_val DAEMON_LIST\\ MASTER, SCHEDD, COLLECTOR'' Furthermore: * It must also hold a valid X509 host certificate. * Must be reachable from everywhere to the TCP port 9619. * The repo for the latest stable release (3.2.1 as of writing) is: * [[https://research.cs.wisc.edu/htcondor/yum/repo.d/htcondor-stable-rhel7.repo]] * Needed RPMs are: * htcondor-ce-bdii, htcondor-ce-client, htcondor-ce-view, htcondor-ce, htcondor-ce-condor * To finalize the configuration one may refer to the official documentation here: [[https://htcondor-ce.readthedocs.io/en/stable/|HTCondor-CE]]. * A few supplementary notes for non OSG people are provided below == VOMS e Grid == Official documentation points to [[https://opensciencegrid.org/docs/security/lcmaps-voms-authentication/|lcmaps-voms-authentication]] which refers to OSG. We could instead refer to the standard configuration of a CREAM-CE, which only needs minor changes:\\ \\ ''ln -s /etc/grid-security/voms-grid-mapfile /etc/grid-security/voms-mapfile\\ cp -a /etc/grid-security/hostcert.pem /etc/grid-security/condorcert.pem\\ cp -a /etc/grid-security/hostkey.pem /etc/grid-security/condorkey.pem\\ chown condor.condor /etc/grid-security/condorcert.pem /etc/grid-security/condorkey.pem\\ '' == GSI and authz == HTCondor-CE relies on argus for authorization. A basic configuration example for argus his detailed below. '' [root@ce02-htc grid-security]# cat /etc/grid-security/gsi-authz.conf\\ globus_mapping /usr/lib64/libgsi_pep_callout.so argus_pep_callout\\ [root@ce02-htc grid-security]#\\ [root@ce02-htc grid-security]# cat /etc/grid-security/gsi-pep-callout-condor.conf\\ pep_ssl_server_capath /etc/grid-security/certificates/\\ pep_ssl_client_cert /etc/grid-security/condorcert.pem\\ pep_ssl_client_key /etc/grid-security/condorkey.pem\\ pep_url https://:8154/authz\\ pep_timeout 30 # seconds\\ xacml_resourceid http:///condor-ce\\ \\ '' == GSI and argus == Refer to the official documentation: [[https://argus-documentation.readthedocs.io/en/stable/|ARGUS]] for general help about installation and configuration. We provide below an example set of configurations for the HTCondor-CE: '' [root@argus ~]# pap-admin lp\\ \\ default (local):\\ \\ resource "http://your.favourite.argus.server/condor-ce" {\\ obligation "http://glite.org/xacml/obligation/local-environment-map" {\\ }\\ \\ action ".*" {\\ rule permit { vo="ops" }\\ rule permit { vo="dteam" }\\ rule permit { vo="alice" }\\ rule permit { vo="atlas" }\\ rule permit { vo="ams02.cern.ch" }\\ rule permit { vo="xenon.biggrid.nl" }\\ }\\ }\\ '' * You can consider installing the argus service on the HTC-CE host itself. This would probably ease early setup. * Be aware that Argus need read/write access to the ''/etc/grid-security/gridmapdir'' directory, which is usually hosted on a shared filesystem. To verify that your Argus service is properly configured to work with your HTC-CE: * Create a valid proxy of a supported VO (i.e.: ''voms-proxy-init --voms cms'') * install an argus pep client on your HTC-CE (''yum install argus-pepcli'') * copy the proxy on the root dir of your HTC-CE as ''user509.pem'' * Execute the following example command (adapt to your case) '' [root@htc-ce-02 ~]# pepcli --pepd https://your.argus.server:8154/authz --keyinfo user509.pem --capath /etc/grid-security/certificates --cert /etc/grid-security/hostcert.pem --key /etc/grid-security/hostkey.pem --resourceid "http://your.argus.server/condor-ce" -a pippo '' On a working setup you should see an output like: '' Resource: http://your.argus.server/condor-ce\\ Decision: Permit\\ Obligation: http://glite.org/xacml/obligation/local-environment-map/posix (caller should resolve POSIX account mapping)\\ Username: cms195\\ Group: cms\\ Secondary Groups: cms\\ '' As a further check you should see that the empty file ''/etc/grid-security/gridmapdir/cms195'' has a static link whose name is derived from the User DN Subject, which can be identified this way: ''\\ [root@argus ~]# ls -li /etc/grid-security/gridmapdir/cms195\\ 383663 -rw-r--r-- 2 root root 0 26 giu 2019 /etc/grid-security/gridmapdir/cms195\\ [root@argus ~]# ls -li /etc/grid-security/gridmapdir/ | grep 383663\\ 383663 -rw-r--r-- 2 root root 0 26 giu 2019 %2fdc%3dch%2fdc%3dcern%2fou%3dorganic%20units%2fou%3dusers%2fcn%3dsurname%2fcn%3d606831%2fcn%3dname%20surname:cms:cmsprd:cms 383663 -rw-r--r-- 2 root root 0 26 giu 2019 cms195\\ '' == condor_mapfile == An entry has to be ADDED to the condor_mapfile to match the certificate DN for the hosts at your site.\\ These are mapped to the value defined by UID_DOMAIN in HTCondor and HTCondor-CE (in our case: "htc_tier1") ''\\ [root@ce02-htc ~]# cat /etc/condor-ce/condor_mapfile\\ GSI "^\/DC\=com\/DC\=DigiCert-Grid\/O=Open Science Grid\/OU\=Services\/CN\=(host\/)?([A-Za-z0-9.\-]*)$"\2@daemon.opensciencegrid.org\\ GSI "^\/DC\=DigiCert-Grid\/DC\=com\/O=Open Science Grid\/OU\=Services\/CN\=(host\/)?([A-Za-z0-9.\-]*)$" \2@daemon.opensciencegrid.org\\ GSI "^\/DC\=org\/DC\=opensciencegrid\/O=Open Science Grid\/OU\=Services\/CN\=(host\/)?([A-Za-z0-9.\-]*)$" \2@daemon.opensciencegrid.org\\ GSI "^\/DC=ch\/DC=cern\/OU=computers\/CN=?([A-Za-z0-9.\-]*)$" \1@cern.ch\\ GSI "^\/C\=IT\/O\=INFN\/OU\=Host\/L=CNAF\/CN\=(host\/)?([A-Za-z0-9.\-]*)$" \1@htc_tier1\\ GSI (.*) GSS_ASSIST_GRIDMAP\\ GSI "(/CN=[-.A-Za-z0-9/= ]+)" \1@unmapped.opensciencegrid.org\\ CLAIMTOBE .* anonymous@claimtobe\\ FS (.*) \1\\ \\ '' === Configuration === The default configuration path is ''/etc/condor-ce/config.d/''. A tool ''condor_ce_config_val'' is provided to inspect configurations, in the same way as ''condor_config_val''. == Examples == * Find //where// and //how// is defined an identifier (Knob) whose name is exactly known: ''[root@ce03-htc ~]# condor_ce_config_val -v HTCONDORCE_VONames\\ HTCONDORCE_VONames = alice, atlas, cdf, cms, dteam, lhcb, virgo\\ # at: /etc/condor-ce/config.d/06-ce-bdii.conf, line 14'' * See names and value for identifiers matching a part: ''[root@ce03-htc ~]# condor_ce_config_val -d HTCONDORCE\\ # Configuration from machine: ce03-htc.cr.cnaf.infn.it\\ \\ # Parameters with names that match HTCONDORCE:\\ HTCONDORCE_BDII_ELECTION = LEADER\\ HTCONDORCE_BDII_LEADER = ce03-htc.cr.cnaf.infn.it\\ HTCONDORCE_CORES = 16 # cores per node\\ [...] '' Most of the predefined values already have reasonable values and there should be no reason to alter them; by individually inspecting the configuration files there are comments providing hints about what should be customized. Worth to mention entries are : * ''HTCONDORCE_VONames = alice, atlas, cdf, cms, dteam, lhcb, virgo'' * ''DAEMON_LIST = $(DAEMON_LIST), CEVIEW, GANGLIAD''\\ from file ''/etc/condor-ce/config.d/05-ce-view.conf''\\ This enables a CE monitoring webtool (CEView) which is visible as http://your.ce.fully.qualified.name:80\\ Atr this point the CE should be able to handle firts job submissions === Testing the CE === * From a User Interface having the ''htcondor-ce-client'' rpm, after generating a valid proxy, the CE can be tested by \\ ''condor_ce_trace ''\\ use the ''--debug'' option to get more details. === BDII === the rpm creates two configuration files and python script: ''\\ [root@ce02-htc bdii]# rpm -ql htcondor-ce-bdii\\ /etc/condor/config.d/50-ce-bdii-defaults.conf\\ /etc/condor/config.d/99-ce-bdii.conf\\ /var/lib/bdii/gip/provider/htcondor-ce-provider\\ '' **Note1:** the path is under /etc/condor/, thus these are HTCondor configurations, not HTCondor-CE; **Note2:** i manually had to define one more knob: ''GLUE2DomainID = $(HTCONDORCE_SiteName)'' Otherwise the script ''htcondor-ce-provider'' would exit with error. (update: fixed in more recent versions) To check that the configuration is formally fine just execute /var/lib/bdii/gip/provider/htcondor-ce-provider; a dump of the glue2 schema shoud appear on ''stdout''. finally, activate the service with ''systemctl enable bdii\\ systemctl start bdii''