===== pcftkvmeX Fresh OS Install - notes =====

These notes are valid on Red Hat-based distributions such as SLC6 and CentOS7. In the following, the shell commands prefaced with ''$'' can be executed by all users, while those prefaced with ''#'' must be executed as root or with increased privileges (''sudo'').

==== Installing AFS ====

[[http://openafs.org/|OpenAFS]] must be installed from scratch, as the default Yum package in the CentOS repositories won't work. Note that this will install OpenAFS in "read-only mode", i.e. users won't be able to authenticate in order to gain write access to the file system. This is sufficient for our purposes.

First of all, install and upgrade all the necessary binaries. You need to import an external repository in order to install [[wp>https://en.wikipedia.org/wiki/Dynamic_Kernel_Module_Support|DKMS]]:

<code>
    # yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
    # yum repolist
</code>

Now you can install all the prerequisites:

<code>
    # yum install -y rpm-build yum-utils make perl libtool bzip2 wget elfutils-devel dkms gcc kernel-devel-$(uname -r) kernel-headers-$(uname -r)
</code>

Make a temp folder in the root user's home and download all the files corresponding to the current stable release of OpenAFS (at the time of writing, 1.8.6).

<code>
    # mkdir ~/temp
    # cd ~/temp
    # wget http://www.openafs.org/dl/openafs/1.8.6/openafs-1.8.6-src.tar.bz2
    # wget http://www.openafs.org/dl/openafs/1.8.6/openafs-1.8.6-doc.tar.bz2
    # wget http://www.openafs.org/dl/openafs/1.8.6/RELNOTES-1.8.6
    # wget http://www.openafs.org/dl/openafs/1.8.6/ChangeLog
</code>

Extract from the source tarball the RPM-making Perl script and use it to compile a ''*.rpm'' file:

<code>
    # tar xf openafs-1.8.6-src.tar.bz2 --strip-components=4 '*/makesrpm.pl'
    # perl makesrpm.pl openafs-1.8.6-src.tar.bz2 openafs-1.8.6-doc.tar.bz2 RELNOTES-1.8.6 ChangeLog
</code>

Check if all necessary dependencies are installed (if not, refer to both the command output and the beginning of this section to find the missing ones):

<code>
    # yum-builddep openafs-1.8.6-1.src.rpm
</code>

Rebuild the RPM in order to perform the included scripts:

<code>
    # rpmbuild --rebuild openafs-1.8.6-1.src.rpm
</code>

Now you can finally install the generated RPMs:

<code>
    # cd ../rpmbuild/RPMS/x86_64/
    # yum install -y openafs-1.8.6-1.el7.x86_64.rpm openafs-client-1.8.6-1.el7.x86_64.rpm openafs-krb5-1.8.6-1.el7.x86_64.rpm dkms-openafs-1.8.6-1.el7.x86_64.rpm
</code>

For the last steps, we need to set the single-board computer as part of the Cern cell:

<code>
    # cd /usr/vice/etc
    # echo "cern.ch" > ThisCell
    # curl -o CellServDB http://afs.web.cern.ch/afs/CellServDB
</code>

We're done! Start and enable the OpenAFS service and check if the ''/afs/'' folder is currently mounted:

<code>
    # systemctl start openafs-client
    # systemctl enable openafs-client
    # ls /afs/
</code>

The last command should return the list of AFS cells (acm-csuf.org, acm.jhu.edu, ...).

==== Installing CVMFS ====

[[https://cernvm.cern.ch/fs/|CVMFS]] is fortunately much easier to install than OpenAFS. First of all, add the repository to Yum:

<code>
    # yum install https://ecsft.cern.ch/dist/cvmfs/cvmfs-release/cvmfs-release-latest.noarch.rpm
    # yum repolist
</code>

Install the package:

<code>
    # yum install -y cvmfs
</code>

Now we need to configure the CVMFS repositories and client profile. Start by invoking the autoconfigure script:

<code>
    # cvmfs_config setup
</code>

Now you need to create the ''/etc/cvmfs/default.local'' plain text file and paste the following lines in it:

<code>
CVMFS_REPOSITORIES=atlas.cern.ch,atlas-condb.cern.ch,grid.cern.ch
CVMFS_CLIENT_PROFILE=single

</code>

Done!

==== Installing the TDAQ driver ====

For convenience, I've included the necessary files for TDAQ 9.2.1 {{progetti:puma:atlas.zip|here}}. For other versions you'll need to retrieve the necessary files from Markus Joos' AFS space. Download and unzip the archive in the root of the file system, i.e. create the following hierarchy:

  * /
    * /ATLAS/
      * /ATLAS/bin/
        * ...
      * /ATLAS/dkmsmod/
        * ...
      * /ATLAS/driver/
        * ...
      * /ATLAS/lib/
        * ...
    * /AFS/
      * ...  
    * ... 

Theoretically, it should suffice to install the RPM file inside ''/ATLAS/dkmsmod/'' to finish the setup. I've found, however, that the success of the operation is highly dependent on the Linux header files available on the machine; the RPM seems to be built for newer kernels. In general, try to install the RPM file:

<code>
    #  yum install -y /ATLAS/dkmsmod/tdaq_drivers-9.2.1-2dkms.noarch.rpm
</code>

If it fails with error ''Bad return status for module build on kernel: KERNEL_VER'' (where ''KERNEL_VER'' is the version number of the current kernel, here 3.10.0-1160.6.1.el7.x86_64), refer to the file ''/var/lib/dkms/tdaq_drivers/9.2.1/build/make.log'' to check the compile errors. In my case, this was reported:

<code>
/var/lib/dkms/tdaq_drivers/9.2.1/build/src/vme_rcc.c:455:72: error: ‘HRTIMER_MODE_SOFT’ undeclared (first use in this function)
</code>

I managed to fix this by modifying the ''/usr/src/kernels/3.10.0-1160.6.1.el7.x86_64/include/linux/hrtimer.h'' header and replacing the following lines:

<code c>
enum hrtimer_mode {
        HRTIMER_MODE_ABS = 0x0,         /* Time value is absolute */
        HRTIMER_MODE_REL = 0x1,         /* Time value is relative to now */
        HRTIMER_MODE_PINNED = 0x02,     /* Timer is bound to CPU */
        HRTIMER_MODE_ABS_PINNED = 0x02,
        HRTIMER_MODE_REL_PINNED = 0x03,
};
</code>

with the following ones, taken by a newer kernel version:

<code c>
enum hrtimer_mode {
	HRTIMER_MODE_ABS	= 0x00,
	HRTIMER_MODE_REL	= 0x01,
	HRTIMER_MODE_PINNED	= 0x02,
	HRTIMER_MODE_SOFT	= 0x04,
	HRTIMER_MODE_HARD	= 0x08,

	HRTIMER_MODE_ABS_PINNED = HRTIMER_MODE_ABS | HRTIMER_MODE_PINNED,
	HRTIMER_MODE_REL_PINNED = HRTIMER_MODE_REL | HRTIMER_MODE_PINNED,

	HRTIMER_MODE_ABS_SOFT	= HRTIMER_MODE_ABS | HRTIMER_MODE_SOFT,
	HRTIMER_MODE_REL_SOFT	= HRTIMER_MODE_REL | HRTIMER_MODE_SOFT,

	HRTIMER_MODE_ABS_PINNED_SOFT = HRTIMER_MODE_ABS_PINNED | HRTIMER_MODE_SOFT,
	HRTIMER_MODE_REL_PINNED_SOFT = HRTIMER_MODE_REL_PINNED | HRTIMER_MODE_SOFT,

	HRTIMER_MODE_ABS_HARD	= HRTIMER_MODE_ABS | HRTIMER_MODE_HARD,
	HRTIMER_MODE_REL_HARD	= HRTIMER_MODE_REL | HRTIMER_MODE_HARD,

	HRTIMER_MODE_ABS_PINNED_HARD = HRTIMER_MODE_ABS_PINNED | HRTIMER_MODE_HARD,
	HRTIMER_MODE_REL_PINNED_HARD = HRTIMER_MODE_REL_PINNED | HRTIMER_MODE_HARD,
};
</code>

You may need to adapt this to your system, modifying other headers in the ''/usr/src/kernels/KERNEL_VER'' folder. You may even need to modify the TDAQ driver source code: the easiest way I've found is to run the ''rpmrebuild'' command

<code>
    #  rpmrebuild -ep /ATLAS/dkmsmod/tdaq_drivers-9.2.1-2dkms.noarch.rpm
</code>

and //while the command is running// open another shell and modify the source files inside ''~/.tmp/rpmrebuild.SOME_NUMBER/work/root''. When you finish modifying the needed source files, close the other shell and exit the ''rpmrebuild'' command (by typing '':wq'' and then confirming with ''y''). The command will return a string like

<code>result: /root/rpmbuild/RPMS/noarch/tdaq_drivers-9.2.1-2dkms.noarch.rpm</code>

which points to an RPM which you can install with ''yum''.