INFN wiki

These notes are valid on Red Hat-based distributions such as SLC6 and CentOS7. In the following, the shell commands prefaced with $ can be executed by all users, while those prefaced with # must be executed as root or with increased privileges (sudo).

OpenAFS must be installed from scratch, as the default Yum package in the CentOS repositories won't work. Note that this will install OpenAFS in "read-only mode", i.e. users won't be able to authenticate in order to gain write access to the file system. This is sufficient for our purposes.

First of all, install and upgrade all the necessary binaries. You need to import an external repository in order to install DKMS:

    # yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
    # yum repolist

Now you can install all the prerequisites:

    # yum install -y rpm-build yum-utils make perl libtool bzip2 wget elfutils-devel dkms gcc kernel-devel-$(uname -r) kernel-headers-$(uname -r)

Make a temp folder in the root user's home and download all the files corresponding to the current stable release of OpenAFS (at the time of writing, 1.8.6).

    # mkdir ~/temp
    # cd ~/temp
    # wget http://www.openafs.org/dl/openafs/1.8.6/openafs-1.8.6-src.tar.bz2
    # wget http://www.openafs.org/dl/openafs/1.8.6/openafs-1.8.6-doc.tar.bz2
    # wget http://www.openafs.org/dl/openafs/1.8.6/RELNOTES-1.8.6
    # wget http://www.openafs.org/dl/openafs/1.8.6/ChangeLog

Extract from the source tarball the RPM-making Perl script and use it to compile a *.rpm file:

    # tar xf openafs-1.8.6-src.tar.bz2 --strip-components=4 '*/makesrpm.pl'
    # perl makesrpm.pl openafs-1.8.6-src.tar.bz2 openafs-1.8.6-doc.tar.bz2 RELNOTES-1.8.6 ChangeLog

Check if all necessary dependencies are installed (if not, refer to both the command output and the beginning of this section to find the missing ones):

    # yum-builddep openafs-1.8.6-1.src.rpm

Rebuild the RPM in order to perform the included scripts:

    # rpmbuild --rebuild openafs-1.8.6-1.src.rpm

Now you can finally install the generated RPMs:

    # cd ../rpmbuild/RPMS/x86_64/
    # yum install -y openafs-1.8.6-1.el7.x86_64.rpm openafs-client-1.8.6-1.el7.x86_64.rpm openafs-krb5-1.8.6-1.el7.x86_64.rpm dkms-openafs-1.8.6-1.el7.x86_64.rpm

For the last steps, we need to set the single-board computer as part of the Cern cell:

    # cd /usr/vice/etc
    # echo "cern.ch" > ThisCell
    # curl -o CellServDB http://afs.web.cern.ch/afs/CellServDB

We're done! Start and enable the OpenAFS service and check if the /afs/ folder is currently mounted:

    # systemctl start openafs-client
    # systemctl enable openafs-client
    # ls /afs/

The last command should return the list of AFS cells (acm-csuf.org, acm.jhu.edu, …).

CVMFS is fortunately much easier to install than OpenAFS. First of all, add the repository to Yum:

    # yum install https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest.noarch.rpm
    # yum repolist

Install the package:

    # yum install -y cvmfs

Now we need to configure the CVMFS repositories and client profile. Start by invoking the autoconfigure script:

    # cvmfs_config setup

Now you need to create the /etc/cvmfs/default.local plain text file and paste the following lines in it:

CVMFS_REPOSITORIES=atlas.cern.ch,atlas-condb.cern.ch,grid.cern.ch
CVMFS_CLIENT_PROFILE=single

Done!

For convenience, I've included the necessary files for TDAQ 9.2.1 here. For other versions you'll need to retrieve the necessary files from Markus Joos' AFS space. Download and unzip the archive in the root of the file system, i.e. create the following hierarchy:

• /
  • /ATLAS/
    • /ATLAS/bin/
      • …
    • /ATLAS/dkmsmod/
      • …
    • /ATLAS/driver/
      • …
    • /ATLAS/lib/
      • …
  • /AFS/
    • …
  • …

Theoretically, it should suffice to install the RPM file inside /ATLAS/dkmsmod/ to finish the setup. I've found, however, that the success of the operation is highly dependent on the Linux header files available on the machine; the RPM seems to be built for newer kernels. In general, try to install the RPM file:

    #  yum install -y /ATLAS/dkmsmod/tdaq_drivers-9.2.1-2dkms.noarch.rpm

If it fails with error Bad return status for module build on kernel: KERNEL_VER (where KERNEL_VER is the version number of the current kernel, here 3.10.0-1160.6.1.el7.x86_64), refer to the file /var/lib/dkms/tdaq_drivers/9.2.1/build/make.log to check the compile errors. In my case, this was reported:

/var/lib/dkms/tdaq_drivers/9.2.1/build/src/vme_rcc.c:455:72: error: ‘HRTIMER_MODE_SOFT’ undeclared (first use in this function)

I managed to fix this by modifying the /usr/src/kernels/3.10.0-1160.6.1.el7.x86_64/include/linux/hrtimer.h header and replacing the following lines:

enum hrtimer_mode {
        HRTIMER_MODE_ABS = 0x0,         /* Time value is absolute */
        HRTIMER_MODE_REL = 0x1,         /* Time value is relative to now */
        HRTIMER_MODE_PINNED = 0x02,     /* Timer is bound to CPU */
        HRTIMER_MODE_ABS_PINNED = 0x02,
        HRTIMER_MODE_REL_PINNED = 0x03,
};

with the following ones, taken by a newer kernel version:

enum hrtimer_mode {
	HRTIMER_MODE_ABS	= 0x00,
	HRTIMER_MODE_REL	= 0x01,
	HRTIMER_MODE_PINNED	= 0x02,
	HRTIMER_MODE_SOFT	= 0x04,
	HRTIMER_MODE_HARD	= 0x08,
 
	HRTIMER_MODE_ABS_PINNED = HRTIMER_MODE_ABS | HRTIMER_MODE_PINNED,
	HRTIMER_MODE_REL_PINNED = HRTIMER_MODE_REL | HRTIMER_MODE_PINNED,
 
	HRTIMER_MODE_ABS_SOFT	= HRTIMER_MODE_ABS | HRTIMER_MODE_SOFT,
	HRTIMER_MODE_REL_SOFT	= HRTIMER_MODE_REL | HRTIMER_MODE_SOFT,
 
	HRTIMER_MODE_ABS_PINNED_SOFT = HRTIMER_MODE_ABS_PINNED | HRTIMER_MODE_SOFT,
	HRTIMER_MODE_REL_PINNED_SOFT = HRTIMER_MODE_REL_PINNED | HRTIMER_MODE_SOFT,
 
	HRTIMER_MODE_ABS_HARD	= HRTIMER_MODE_ABS | HRTIMER_MODE_HARD,
	HRTIMER_MODE_REL_HARD	= HRTIMER_MODE_REL | HRTIMER_MODE_HARD,
 
	HRTIMER_MODE_ABS_PINNED_HARD = HRTIMER_MODE_ABS_PINNED | HRTIMER_MODE_HARD,
	HRTIMER_MODE_REL_PINNED_HARD = HRTIMER_MODE_REL_PINNED | HRTIMER_MODE_HARD,
};

You may need to adapt this to your system, modifying other headers in the /usr/src/kernels/KERNEL_VER folder. You may even need to modify the TDAQ driver source code: the easiest way I've found is to run the rpmrebuild command

    #  rpmrebuild -ep /ATLAS/dkmsmod/tdaq_drivers-9.2.1-2dkms.noarch.rpm

and while the command is running open another shell and modify the source files inside ~/.tmp/rpmrebuild.SOME_NUMBER/work/root. When you finish modifying the needed source files, close the other shell and exit the rpmrebuild command (by typing :wq and then confirming with y). The command will return a string like

result: /root/rpmbuild/RPMS/noarch/tdaq_drivers-9.2.1-2dkms.noarch.rpm

which points to an RPM which you can install with yum.