User Tools

Site Tools


Sidebar

progetti:cloud-areapd:ced-c:install_and_configure_compute_nodes_draft

Installation and Configuration of OpenStack Compute Node

Author:

  • Paolo E. Mazzon (DEI)
  • Matteo Menguzzato (DFA)
  • Gianpietro Sella (DISC)

Prerequisites

At least one node with:

  • Updated CentOS 7 (7.1)
  • Make sure that yum autoupdate is disabled
[root@cld-blu-11 ~]# grep ENA /etc/sysconfig/yum-autoupdate
# ENABLED
ENABLED="false"
  • At least 20GB HD for operating system and OpenStack software and related log files
  • Dedicated storage mounted on /var/lib/nova/instances where to store the instance images (particularly important to get live migration).
  • SELinux configured as "Disabled" (/etc/selinux/config)
  • EPEL 7
  • A MySQL (possibly a HA cluster) endpoint each OpenStack service can connect to (in this guide we're using our MySQL Percona cluster's IP 192.168.60.180)
  • A HAProxy/Keepalived cluster to use for load-balancing and Virtual IP (in this guide we're using the IP 192.168.60.180 for mgmt net and 90.147.143.10 for public net)
  • Disable NetworkManager
  • Installed CA INFN certificate on both nodes
  • add user "nova" to group "disk" and reboot node
[root@cld-blu-11 ~]# ll /etc/grid-security/chain.pem
-rw-r--r-- 1 root root 5607  9 apr 19.01 chain.pem
  • Installed and active libvirt
yum -y install libvirt
systemctl start libvirtd
systemctl enable libvirtd
  • Activated virtualization on CPU (can be toggled in the BIOS menu):
cat /proc/cpuinfo | grep vmx
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm dts tpr_shadow vnmi flexpriority
 
lsmod |grep kvm
kvm_intel              54285  12 
kvm                   332980  1 kvm_intel
 
lscpu |grep -i virtu
Virtualization:        VT-x

Note: kvm_intel can be substituted by kvm_amd, and VT-x can be substituted by AMD-V.

Naming conventions and networking assumptions

We assume that the compute node has the following network setup:

  • It has two network interface connected to two different networks: management network and data network
  • Management network is: 192.168.60.0/24
  • Data network is: 192.168.61.0/24
  • Node's IP are: 192.168.60.XYZ and 192.168.61.XYZ

Further pre-requisite on data network interface

In the net-interface configuration script for data network (something like /etc/sysconfig/network-scripts/ifcfg-ethX) put the following parameter:

MTU="9000"

IPTables configuration

Execute the following commands:

while read i
do
   firewall-cmd --add-port=${i}/tcp
   firewall-cmd --permanent --add-port=${i}/tcp
done << EOF
5900-5999
16509
49152-49261
EOF
 
# 5900-5999    VNC's TCP ports
# 16509        libvirtd's TCP ports
# 49152-49261  libvirtd's ephemeral ports
 
# permit ntpd's udp communications
firewall-cmd --add-port=123/udp
firewall-cmd --permanent --add-port=123/udp
firewall-cmd --permanent --direct --add-rule ipv4 filter INPUT 0 -p gre -j ACCEPT

Naming conventions and networking assumptions

We assume that the compute node has the following setup:

  • It has 2 network interfaces connected to two different networks: management network, Data network
  • Management network is: 192.168.60.0/24
  • Data network is: 192.168.61.0/24
  • The node is named: cld-blu-NN.cloud.pd.infn.it (192.168.60.XYZ, 192.168.61.XYZ)
  • In this guide the controller's VIP on the management network is needed: 192.168.60.180
  • In this guide the MySQL cluster's VIP on the management network is needed: 192.168.60.180
  • In this guide the controller's public IP is needed: 90.147.143.10

Install software

Install repo and Nova and Neutron's packages, and update iproute to support network namespaces:

yum -y install https://repos.fedorapeople.org/repos/openstack/EOL/openstack-icehouse/rdo-release-icehouse-4.noarch.rpm
yum -y install openstack-nova-compute openstack-utils openstack-neutron-openvswitch openstack-neutron-ml2 sysfsutils
yum -y update iproute

Preliminary networking setup

sed -i 's+^net\.ipv4\.conf\.default\.rp_filter+#net\.ipv4\.conf\.default\.rp_filter+' /etc/sysctl.conf
sed -i 's+^net\.ipv4\.conf\.all\.rp_filter+#net\.ipv4\.conf\.all\.rp_filter+' /etc/sysctl.conf
cat << EOF >> /etc/sysctl.conf
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0
net.bridge.bridge-nf-call-arptables=1
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
EOF
sysctl -p
systemctl restart network

Configure Nova

nova.conf

openstack-config --set /etc/nova/nova.conf database connection "mysql://nova:<NOVA_DB_PASS>@192.168.60.180:5306/nova"
openstack-config --set /etc/nova/nova.conf DEFAULT auth_strategy keystone
openstack-config --set /etc/nova/nova.conf keystone_authtoken auth_host 192.168.60.180
openstack-config --set /etc/nova/nova.conf keystone_authtoken auth_protocol http
openstack-config --set /etc/nova/nova.conf keystone_authtoken auth_port 35357
openstack-config --set /etc/nova/nova.conf keystone_authtoken admin_user nova
openstack-config --set /etc/nova/nova.conf keystone_authtoken admin_tenant_name services
openstack-config --set /etc/nova/nova.conf keystone_authtoken admin_password NOVA_PASS
openstack-config --set /etc/nova/nova.conf DEFAULT rpc_backend nova.openstack.common.rpc.impl_kombu
openstack-config --set /etc/nova/nova.conf DEFAULT rabbit_hosts 192.168.60.152:5672,192.168.60.153:5672
openstack-config --set /etc/nova/nova.conf libvirt live_migration_flag VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE
 
# Change the following IP with the actual IP of the current compute node on the management network
openstack-config --set /etc/nova/nova.conf DEFAULT my_ip 192.168.60.XYZ
openstack-config --set /etc/nova/nova.conf DEFAULT vnc_enabled True
# vncserver_listen MUST be 0.0.0.0 otherwise the live migration won't work correctly
# (http://docs.openstack.org/havana/config-reference/content/configuring-openstack-compute-basics.html#setting-flags-in-nova-conf-file)
openstack-config --set /etc/nova/nova.conf DEFAULT vncserver_listen 0.0.0.0
 
# Change the following IP with the actual IP of the current compute node on the management network
openstack-config --set /etc/nova/nova.conf DEFAULT vncserver_proxyclient_address 192.168.60.XYZ
openstack-config --set /etc/nova/nova.conf DEFAULT novncproxy_base_url http://90.147.143.10:6080/vnc_auto.html
openstack-config --set /etc/nova/nova.conf DEFAULT glance_host 192.168.60.180
openstack-config --set /etc/nova/nova.conf DEFAULT compute_driver nova.virt.libvirt.LibvirtDriver
openstack-config --set /etc/nova/nova.conf DEFAULT api_paste_config /etc/nova/api-paste.ini
openstack-config --set /etc/nova/nova.conf DEFAULT network_api_class nova.network.neutronv2.api.API
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_url http://192.168.60.180:9696
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_auth_strategy keystone
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_admin_tenant_name services
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_admin_username neutron
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_admin_password NEUTRON_PASS
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_admin_auth_url http://192.168.60.180:35357/v2.0
openstack-config --set /etc/nova/nova.conf DEFAULT linuxnet_interface_driver nova.network.linux_net.LinuxOVSInterfaceDriver
openstack-config --set /etc/nova/nova.conf DEFAULT security_group_api neutron
openstack-config --set /etc/nova/nova.conf DEFAULT firewall_driver nova.virt.firewall.NoopFirewallDriver
# the following 3 lines enable admin's password inject
openstack-config --set /etc/nova/nova.conf libvirt inject_password true
openstack-config --set /etc/nova/nova.conf libvirt inject_key true
openstack-config --set /etc/nova/nova.conf libvirt inject_partition -1
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_metadata_proxy_shared_secret METADATA_PASS
openstack-config --set /etc/nova/nova.conf DEFAULT rabbit_ha_queues True
openstack-config --set /etc/nova/nova.conf DEFAULT cpu_allocation_ratio 4.0
openstack-config --set /etc/nova/nova.conf DEFAULT allow_resize_to_same_host true
 
 
# this is a temporary workaround untill we understand a problem of cpu not compatible when live-migrating VMs
openstack-config --set /etc/nova/nova.conf libvirt cpu_mode custom
openstack-config --set /etc/nova/nova.conf libvirt cpu_model kvm64

api-paste.ini

while read i
do
   openstack-config --set /etc/nova/api-paste.ini ${i}
done << EOF
filter:authtoken paste.filter_factory keystoneclient.middleware.auth_token:filter_factory
filter:authtoken auth_host 192.168.60.180
filter:authtoken auth_uri http://192.168.60.180:5000
filter:authtoken admin_tenant_name services
filter:authtoken admin_user nova
filter:authtoken admin_password NOVA_PASS
EOF

Configure LibVirt to support Live Migration

Turn OFF the libvirtd daemon :

systemctl stop libvirtd

Execute:

cat << EOF >> /etc/libvirt/libvirtd.conf
listen_tls = 0
listen_tcp = 1
auth_tcp = "none"
EOF

and

cat << EOF >> /etc/sysconfig/libvirtd
LIBVIRTD_ARGS="--listen"
EOF

Modify qemu.conf:

cat << EOF >> /etc/libvirt/qemu.conf 
user="nova"
group="nova"
dynamic_ownership = 0
EOF

Configure Neutron's agents

As in the compute node the Neutron's L2 agent is running, some Neutron's configuration files need to be customized.

neutron.conf

openstack-config --set /etc/neutron/neutron.conf DEFAULT core_plugin ml2
openstack-config --set /etc/neutron/neutron.conf DEFAULT service_plugins router
openstack-config --set /etc/neutron/neutron.conf DEFAULT api_paste_config /etc/neutron/api-paste.ini
openstack-config --set /etc/neutron/neutron.conf DEFAULT rpc_backend neutron.openstack.common.rpc.impl_kombu
openstack-config --set /etc/neutron/neutron.conf DEFAULT rabbit_hosts 192.168.60.152:5672,192.168.60.153:5672
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_host 192.168.60.180
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken admin_user neutron
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken admin_password NEUTRON_PASS
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_url http://192.168.60.180:35357/v2.0
openstack-config --set /etc/neutron/neutron.conf DEFAULT auth_strategy keystone
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken admin_tenant_name services
openstack-config --set /etc/neutron/neutron.conf agent root_helper "sudo neutron-rootwrap /etc/neutron/rootwrap.conf"
openstack-config --set /etc/neutron/neutron.conf database connection "mysql://neutron:<NEUTRON_DB_PWD>@192.168.60.180:4306/neutron"
openstack-config --set /etc/neutron/neutron.conf DEFAULT dhcp_agents_per_network 2
openstack-config --set /etc/neutron/neutron.conf DEFAULT dhcp_lease_duration 86400
openstack-config --set /etc/neutron/neutron.conf DEFAULT rabbit_ha_queues True
openstack-config --set /etc/neutron/neutron.conf DEFAULT agent_down_time 75 
openstack-config --set /etc/neutron/neutron.conf agent report_interval 30 
openstack-config --set /etc/neutron/neutron.conf filter:authtoken auth_port 35357

ml2_conf.ini

for local_ip parameter use the correct value

while read i
do
   openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ${i}
done << EOF
ml2 type_drivers gre
ml2 tenant_network_types gre
ml2 mechanism_drivers openvswitch
ml2_type_gre tunnel_id_ranges 1:1000
securitygroup firewall_driver neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
securitygroup enable_security_group True
ovs tenant_network_type gre
ovs tunnel_id_ranges 1:1000
ovs local_ip 192.168.61.XYZ
ovs enable_tunneling True
ovs integration_bridge br-int
ovs tunnel_bridge br-tun
EOF
 
ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini

(Neutron L2 agent's) api-paste.ini

openstack-config --set /etc/neutron/api-paste.ini filter:authtoken paste.filter_factory keystoneclient.middleware.auth_token:filter_factory
openstack-config --set /etc/neutron/api-paste.ini filter:authtoken auth_host 192.168.60.180
openstack-config --set /etc/neutron/api-paste.ini filter:authtoken admin_tenant_name services 
openstack-config --set /etc/neutron/api-paste.ini filter:authtoken admin_user neutron 
openstack-config --set /etc/neutron/api-paste.ini filter:authtoken admin_password NEUTRON_PASS

Bridge creation and start of the services

Open vSwitch start and bridge creation

systemctl start openvswitch
systemctl enable openvswitch
ovs-vsctl add-br br-int

L2 Agent configuration:

sed -i 's,plugins/openvswitch/ovs_neutron_plugin.ini,plugin.ini,g' /usr/lib/systemd/system/neutron-openvswitch-agent.service
systemctl daemon-reload

Mute neutron-openvswitch-agent eccessive logging:

The openvswitch service execs every two seconds the command ovs-ofctl dump-flows br-int table=22 through sudo. The execution of the command itself produces 43200 entries a day in /var/log/secure. To mute the logging we add the !syslog directive to /etc/sudoers.d/neutron file:

sed -i 's/Defaults:neutron !requiretty$/Defaults:neutron !requiretty, !syslog/' /etc/sudoers.d/neutron

Start all the services:

systemctl start neutron-openvswitch-agent
systemctl enable neutron-openvswitch-agent
systemctl start libvirtd
systemctl start messagebus
systemctl start openstack-nova-compute
systemctl enable openstack-nova-compute

Check all

When done, log into the controller node, or wherever you've installed the Openstack CLI and copied the keystone_admin.sh into (which was created in the controller node installation procedure). Execute the commands:

[root@cld-blu-03 ~]# neutron agent-list
 
+--------------------------------------+--------------------+-----------------------------+-------+----------------+
| id                                   | agent_type         | host                        | alive | admin_state_up |
+--------------------------------------+--------------------+-----------------------------+-------+----------------+
| 22cf4da6-3925-4a31-ba95-73e3332b1a36 | Open vSwitch agent | cld-blu-NN.cloud.pd.infn.it | :-)   | True           |
| 2d6aed8f-5a6d-46b1-8ce8-c07f4022d1d5 | DHCP agent         | cld-blu-03.cloud.pd.infn.it | :-)   | True           |
| 4d01a60a-b25c-4530-9860-c37490069e5e | Metadata agent     | cld-blu-03.cloud.pd.infn.it | :-)   | True           |
| 662f5529-675a-41e8-9490-4f0149edb6ce | Open vSwitch agent | cld-blu-03.cloud.pd.infn.it | :-)   | True           |
| c4c512b8-68cc-4deb-8451-d4c3d827809c | L3 agent           | cld-blu-03.cloud.pd.infn.it | :-)   | True           |
+--------------------------------------+--------------------+-----------------------------+-------+----------------+
 
 
[root@cld-blu-03 ~]# nova service-list
+------------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+
| Binary           | Host                        | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+------------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+
| nova-consoleauth | cld-blu-03.cloud.pd.infn.it | internal | enabled | up    | 2015-03-27T10:26:58.000000 | -               |
| nova-conductor   | cld-blu-03.cloud.pd.infn.it | internal | enabled | up    | 2015-03-27T10:26:58.000000 | -               |
| nova-scheduler   | cld-blu-03.cloud.pd.infn.it | internal | enabled | up    | 2015-03-27T10:27:00.000000 | -               |
| nova-cert        | cld-blu-03.cloud.pd.infn.it | internal | enabled | up    | 2015-03-27T10:26:58.000000 | -               |
| nova-cert        | cld-blu-04.cloud.pd.infn.it | internal | enabled | up    | 2015-03-27T10:26:58.000000 | -               |
| nova-consoleauth | cld-blu-04.cloud.pd.infn.it | internal | enabled | up    | 2015-03-27T10:26:56.000000 | -               |
| nova-scheduler   | cld-blu-04.cloud.pd.infn.it | internal | enabled | up    | 2015-03-27T10:27:01.000000 | -               |
| nova-conductor   | cld-blu-04.cloud.pd.infn.it | internal | enabled | up    | 2015-03-27T10:27:04.000000 | -               |
| nova-compute     | cld-blu-NN.cloud.pd.infn.it | nova     | enabled | up    | 2015-03-27T10:27:02.000000 | -               |
+------------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+

Add SSH passwordless access from Compute node to virtual instances

This is needed to allow nova to resize virtual instances. Execute the following commands:

usermod -s /bin/bash nova
mkdir -p -m 700 ~nova/.ssh
chown nova.nova ~nova/.ssh
cd ~nova/.ssh
scp cld-blu-03:/var/lib/nova/.ssh/* .
chown nova.nova * 

Optional: Configure Nova Compute for SSL

openstack-config --set /etc/nova/nova.conf keystone_authtoken auth_host cloud.cedc.csia.unipd.it
openstack-config --set /etc/nova/nova.conf keystone_authtoken auth_protocol https
openstack-config --set /etc/nova/nova.conf keystone_authtoken cafile /etc/grid-security/chain.pem
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_ca_certificates_file /etc/grid-security/chain.pem
openstack-config --set /etc/nova/nova.conf DEFAULT cinder_ca_certificates_file /etc/grid-security/chain.pem
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_admin_auth_url https://cloud.cedc.csia.unipd.it:35357/v2.0
openstack-config --set /etc/nova/nova.conf DEFAULT novncproxy_base_url https://cloud.cedc.csia.unipd.it:6080/vnc_auto.html
 
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_host cloud.cedc.csia.unipd.it
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_url https://cloud.cedc.csia.unipd.it:35357/v2.0
openstack-config --set /etc/neutron/neutron.conf keystone_authtoken auth_protocol https
openstack-config --set /etc/neutron/neutron.conf DEFAULT ssl_ca_file /etc/grid-security/chain.pem
openstack-config --set /etc/neutron/api-paste.ini filter:authtoken auth_host cloud.cedc.csia.unipd.it
 
openstack-config --set /etc/nova/api-paste.ini filter:authtoken auth_host cloud.cedc.csia.unipd.it
openstack-config --set /etc/nova/api-paste.ini filter:authtoken auth_protocol https
 
 
openstack-config --set /etc/nova/nova.conf DEFAULT glance_host cloud.cedc.csia.unipd.it
openstack-config --set /etc/nova/nova.conf DEFAULT glance_protocol https
openstack-config --set /etc/nova/nova.conf DEFAULT glance_api_servers https://cloud.cedc.csia.unipd.it:9292
openstack-config --set /etc/nova/nova.conf DEFAULT ssl_ca_file /etc/grid-security/chain.pem
#openstack-config --set /etc/nova/nova.conf ssl ca_file /etc/grid-security/chain.pem
openstack-config --set /etc/nova/nova.conf DEFAULT glance_api_insecure true
 
openstack-config --set /etc/nova/nova.conf DEFAULT neutron_url https://cloud.cedc.csia.unipd.it:9696

Restart L2 agent and Nova Compute

systemctl restart openstack-nova-compute
systemctl restart neutron-openvswitch-agent

Fix metadata agent

To address this bug, apply this patch, or follow the instructions below:

curl -o agent.py https://raw.githubusercontent.com/CloudPadovana/SSL_Patches/master/agent.py
mv /usr/lib/python2.6/site-packages/neutron/agent/metadata/agent.py /usr/lib/python2.6/site-packages/neutron/agent/metadata/agent.py.bak
cp agent.py /usr/lib/python2.6/site-packages/neutron/agent/metadata/agent.py
 
service openstack-nova-compute restart
service neutron-openvswitch-agent restart
progetti/cloud-areapd/ced-c/install_and_configure_compute_nodes_draft.txt · Last modified: 2015/10/30 12:33 by sella@infn.it