progetti:cloud-areapd:ced-c:install_and_configure_compute_nodes_draft
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
progetti:cloud-areapd:ced-c:install_and_configure_compute_nodes_draft [2015/10/14 13:09] – [Mute neutron-openvswitch-agent eccessive logging] mazzon@infn.it | progetti:cloud-areapd:ced-c:install_and_configure_compute_nodes_draft [2015/10/30 12:33] (current) – [Configure Neutron's agents] sella@infn.it | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ======= Installation and Configuration of OpenStack Compute Node ======= | ||
+ | Author: | ||
+ | * Paolo E. Mazzon (DEI) | ||
+ | * Matteo Menguzzato (DFA) | ||
+ | * Gianpietro Sella (DISC) | ||
+ | ===== Prerequisites ===== | ||
+ | * [[http:// | ||
+ | |||
+ | At least one node with: | ||
+ | * Updated CentOS 7 (7.1) | ||
+ | * Make sure that yum autoupdate is disabled | ||
+ | <code bash> | ||
+ | [root@cld-blu-11 ~]# grep ENA / | ||
+ | # ENABLED | ||
+ | ENABLED=" | ||
+ | </ | ||
+ | * At least 20GB HD for operating system and OpenStack software and related log files | ||
+ | * Dedicated storage mounted on ''/ | ||
+ | * SELinux configured as " | ||
+ | * EPEL 7 | ||
+ | * A MySQL (possibly a HA cluster) endpoint each OpenStack service can connect to (in this guide we're using our MySQL Percona cluster' | ||
+ | * A HAProxy/ | ||
+ | * Disable NetworkManager | ||
+ | * Installed CA INFN certificate on both nodes | ||
+ | * add user " | ||
+ | <code bash> | ||
+ | [root@cld-blu-11 ~]# ll / | ||
+ | -rw-r--r-- 1 root root 5607 9 apr 19.01 chain.pem | ||
+ | </ | ||
+ | * Installed and active libvirt | ||
+ | <code bash> | ||
+ | yum -y install libvirt | ||
+ | systemctl start libvirtd | ||
+ | systemctl enable libvirtd | ||
+ | </ | ||
+ | * Activated virtualization on CPU (can be toggled in the BIOS menu): | ||
+ | <code bash> | ||
+ | cat / | ||
+ | flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm dts tpr_shadow vnmi flexpriority | ||
+ | |||
+ | lsmod |grep kvm | ||
+ | kvm_intel | ||
+ | kvm | ||
+ | |||
+ | lscpu |grep -i virtu | ||
+ | Virtualization: | ||
+ | |||
+ | </ | ||
+ | Note: '' | ||
+ | ===== Naming conventions and networking assumptions ===== | ||
+ | We assume that the compute node has the following network setup: | ||
+ | * It has two network interface connected to two different networks: **management network** and **data network** | ||
+ | * **Management network** is: '' | ||
+ | * **Data network** is: '' | ||
+ | * Node's IP are: '' | ||
+ | |||
+ | ==== Further pre-requisite on data network interface ==== | ||
+ | In the net-interface configuration script for data network (something like ''/ | ||
+ | <code bash> | ||
+ | MTU=" | ||
+ | </ | ||
+ | ===== IPTables configuration ===== | ||
+ | Execute the following commands: | ||
+ | <code bash> | ||
+ | while read i | ||
+ | do | ||
+ | | ||
+ | | ||
+ | done << EOF | ||
+ | 5900-5999 | ||
+ | 16509 | ||
+ | 49152-49261 | ||
+ | EOF | ||
+ | |||
+ | # 5900-5999 | ||
+ | # 16509 libvirtd' | ||
+ | # 49152-49261 | ||
+ | |||
+ | # permit ntpd's udp communications | ||
+ | firewall-cmd --add-port=123/ | ||
+ | firewall-cmd --permanent --add-port=123/ | ||
+ | firewall-cmd --permanent --direct --add-rule ipv4 filter INPUT 0 -p gre -j ACCEPT | ||
+ | </ | ||
+ | ===== Naming conventions and networking assumptions ===== | ||
+ | We assume that the compute node has the following setup: | ||
+ | * It has 2 network interfaces connected to two different networks: **management network**, **Data network** | ||
+ | * **Management network** is: '' | ||
+ | * **Data network** is: 192.168.61.0/ | ||
+ | * The node is named: '' | ||
+ | * In this guide the controller' | ||
+ | * In this guide the MySQL cluster' | ||
+ | * In this guide the controller' | ||
+ | ===== Install software ===== | ||
+ | Install repo and Nova and Neutron' | ||
+ | <code bash> | ||
+ | yum -y install https:// | ||
+ | yum -y install openstack-nova-compute openstack-utils openstack-neutron-openvswitch openstack-neutron-ml2 sysfsutils | ||
+ | yum -y update iproute | ||
+ | </ | ||
+ | |||
+ | |||
+ | ===== Preliminary networking setup ===== | ||
+ | <code bash> | ||
+ | sed -i ' | ||
+ | sed -i ' | ||
+ | cat << EOF >> / | ||
+ | net.ipv4.conf.all.rp_filter=0 | ||
+ | net.ipv4.conf.default.rp_filter=0 | ||
+ | net.bridge.bridge-nf-call-arptables=1 | ||
+ | net.bridge.bridge-nf-call-iptables=1 | ||
+ | net.bridge.bridge-nf-call-ip6tables=1 | ||
+ | EOF | ||
+ | sysctl -p | ||
+ | systemctl restart network | ||
+ | </ | ||
+ | ===== Configure Nova ===== | ||
+ | **nova.conf** | ||
+ | <code bash> | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | |||
+ | # Change the following IP with the actual IP of the current compute node on the management network | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | # vncserver_listen MUST be 0.0.0.0 otherwise the live migration won't work correctly | ||
+ | # (http:// | ||
+ | openstack-config --set / | ||
+ | |||
+ | # Change the following IP with the actual IP of the current compute node on the management network | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | # the following 3 lines enable admin' | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | |||
+ | |||
+ | # this is a temporary workaround untill we understand a problem of cpu not compatible when live-migrating VMs | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | |||
+ | </ | ||
+ | **api-paste.ini** | ||
+ | <code bash> | ||
+ | while read i | ||
+ | do | ||
+ | | ||
+ | done << EOF | ||
+ | filter: | ||
+ | filter: | ||
+ | filter: | ||
+ | filter: | ||
+ | filter: | ||
+ | filter: | ||
+ | EOF | ||
+ | </ | ||
+ | |||
+ | |||
+ | ===== Configure LibVirt to support Live Migration ===== | ||
+ | Turn OFF the '' | ||
+ | <code bash> | ||
+ | systemctl stop libvirtd | ||
+ | </ | ||
+ | Execute: | ||
+ | <code bash> | ||
+ | cat << EOF >> / | ||
+ | listen_tls = 0 | ||
+ | listen_tcp = 1 | ||
+ | auth_tcp = " | ||
+ | EOF | ||
+ | </ | ||
+ | and | ||
+ | <code bash> | ||
+ | cat << EOF >> / | ||
+ | LIBVIRTD_ARGS=" | ||
+ | EOF | ||
+ | </ | ||
+ | Modify '' | ||
+ | <code bash> | ||
+ | cat << EOF >> / | ||
+ | user=" | ||
+ | group=" | ||
+ | dynamic_ownership = 0 | ||
+ | EOF | ||
+ | </ | ||
+ | ===== Configure Neutron' | ||
+ | As in the compute node the Neutron' | ||
+ | |||
+ | **neutron.conf** | ||
+ | <code bash> | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | </ | ||
+ | |||
+ | **ml2_conf.ini** | ||
+ | |||
+ | for local_ip parameter use the correct value | ||
+ | <code bash> | ||
+ | while read i | ||
+ | do | ||
+ | | ||
+ | done << EOF | ||
+ | ml2 type_drivers gre | ||
+ | ml2 tenant_network_types gre | ||
+ | ml2 mechanism_drivers openvswitch | ||
+ | ml2_type_gre tunnel_id_ranges 1:1000 | ||
+ | securitygroup firewall_driver neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver | ||
+ | securitygroup enable_security_group True | ||
+ | ovs tenant_network_type gre | ||
+ | ovs tunnel_id_ranges 1:1000 | ||
+ | ovs local_ip 192.168.61.XYZ | ||
+ | ovs enable_tunneling True | ||
+ | ovs integration_bridge br-int | ||
+ | ovs tunnel_bridge br-tun | ||
+ | EOF | ||
+ | |||
+ | ln -s / | ||
+ | </ | ||
+ | (Neutron L2 agent' | ||
+ | <code bash> | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | </ | ||
+ | ===== Bridge creation and start of the services ===== | ||
+ | Open vSwitch start and bridge creation | ||
+ | <code bash> | ||
+ | systemctl start openvswitch | ||
+ | systemctl enable openvswitch | ||
+ | ovs-vsctl add-br br-int | ||
+ | </ | ||
+ | L2 Agent configuration: | ||
+ | <code bash> | ||
+ | sed -i ' | ||
+ | systemctl daemon-reload | ||
+ | </ | ||
+ | Mute neutron-openvswitch-agent eccessive logging: | ||
+ | |||
+ | The openvswitch service execs every two seconds the command '' | ||
+ | <code bash> | ||
+ | sed -i ' | ||
+ | </ | ||
+ | Start all the services: | ||
+ | <code bash> | ||
+ | systemctl start neutron-openvswitch-agent | ||
+ | systemctl enable neutron-openvswitch-agent | ||
+ | systemctl start libvirtd | ||
+ | systemctl start messagebus | ||
+ | systemctl start openstack-nova-compute | ||
+ | systemctl enable openstack-nova-compute | ||
+ | </ | ||
+ | ===== Check all ===== | ||
+ | When done, log into the controller node, or wherever you've installed the Openstack CLI and copied the '' | ||
+ | <code bash> | ||
+ | [root@cld-blu-03 ~]# neutron agent-list | ||
+ | |||
+ | +--------------------------------------+--------------------+-----------------------------+-------+----------------+ | ||
+ | | id | agent_type | ||
+ | +--------------------------------------+--------------------+-----------------------------+-------+----------------+ | ||
+ | | 22cf4da6-3925-4a31-ba95-73e3332b1a36 | Open vSwitch agent | cld-blu-NN.cloud.pd.infn.it | :-) | True | | ||
+ | | 2d6aed8f-5a6d-46b1-8ce8-c07f4022d1d5 | DHCP agent | cld-blu-03.cloud.pd.infn.it | :-) | True | | ||
+ | | 4d01a60a-b25c-4530-9860-c37490069e5e | Metadata agent | cld-blu-03.cloud.pd.infn.it | :-) | True | | ||
+ | | 662f5529-675a-41e8-9490-4f0149edb6ce | Open vSwitch agent | cld-blu-03.cloud.pd.infn.it | :-) | True | | ||
+ | | c4c512b8-68cc-4deb-8451-d4c3d827809c | L3 agent | cld-blu-03.cloud.pd.infn.it | :-) | True | | ||
+ | +--------------------------------------+--------------------+-----------------------------+-------+----------------+ | ||
+ | |||
+ | |||
+ | [root@cld-blu-03 ~]# nova service-list | ||
+ | +------------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+ | ||
+ | | Binary | ||
+ | +------------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+ | ||
+ | | nova-consoleauth | cld-blu-03.cloud.pd.infn.it | internal | enabled | up | 2015-03-27T10: | ||
+ | | nova-conductor | ||
+ | | nova-scheduler | ||
+ | | nova-cert | ||
+ | | nova-cert | ||
+ | | nova-consoleauth | cld-blu-04.cloud.pd.infn.it | internal | enabled | up | 2015-03-27T10: | ||
+ | | nova-scheduler | ||
+ | | nova-conductor | ||
+ | | nova-compute | ||
+ | +------------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+ | ||
+ | </ | ||
+ | |||
+ | ===== Add SSH passwordless access from Compute node to virtual instances ===== | ||
+ | This is needed to allow nova to resize virtual instances. Execute the following commands: | ||
+ | <code bash> | ||
+ | usermod -s /bin/bash nova | ||
+ | mkdir -p -m 700 ~nova/.ssh | ||
+ | chown nova.nova ~nova/.ssh | ||
+ | cd ~nova/.ssh | ||
+ | scp cld-blu-03:/ | ||
+ | chown nova.nova * | ||
+ | </ | ||
+ | |||
+ | ===== Optional: Configure Nova Compute for SSL ===== | ||
+ | <code bash> | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | |||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | |||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | |||
+ | |||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | openstack-config --set / | ||
+ | # | ||
+ | openstack-config --set / | ||
+ | |||
+ | openstack-config --set / | ||
+ | </ | ||
+ | Restart L2 agent and Nova Compute | ||
+ | <code bash> | ||
+ | systemctl restart openstack-nova-compute | ||
+ | systemctl restart neutron-openvswitch-agent | ||
+ | |||
+ | </ | ||
+ | === Fix metadata agent === | ||
+ | To address this [[https:// | ||
+ | <code bash> | ||
+ | curl -o agent.py https:// | ||
+ | mv / | ||
+ | cp agent.py / | ||
+ | |||
+ | service openstack-nova-compute restart | ||
+ | service neutron-openvswitch-agent restart | ||
+ | |||
+ | </ | ||
+ | |||
+ | === === |