======= Install and configure Controller nodes ====== Authors: * Paolo E. Mazzon (DEI) * Matteo Menguzzato (INFN Padova) * Gianpietro Sella (DISC) ===== Reference Links ===== * [[progetti:cloud-areapd:keystone-glance_high_availability:openstack_ha:temp|Installation and Configuration of OpenStack Controller Node (Havana version)]] * [[http://docs.openstack.org/icehouse/install-guide/install/yum/content/|Manual OpenStack installation guide]] * [[https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux_OpenStack_Platform/4/html/Installation_and_Configuration_Guide/part-Installing_OpenStack_Manually.html|Manual OpenStack installation guide from RedHat]] * [[progetti:cloud-areapd:ced-c:install_ad_setup_mysql_percona_xtradb_-_haproxy_-_keepalived|Percona - HAproxy - Keepalived configuration]] (This link refers to a configuration for the highly available MySQL cluster. Below we will explain how to configure HAProxy also for the OpenStack services.) * [[http://www.mirantis.com/blog/ha-platform-components-mysql-rabbitmq/|Mirantis' guidelines on OpenStack HA]] * [[http://www.rabbitmq.com/clustering.html|Clustering RabbitMQ]] ===== Naming conventions and networking assumptions ===== We assume that the controller nodes have the following setup: * They have 1 network interfaces connected to 1 networks: **management network (em1)** * **Management network** is: ''192.168.60.0/24'' * First node is named: ''cld-blu-03.cloud.pd.infn.it'' (''192.168.60.152'') * Second node is named: ''cld-blu-04.cloud.pd.infn.it'' (''192.168.60.153'') ===== Prerequisites ===== Two nodes with: * Updated CentOS7 * Make sure that yum autoupdate is disabled [root@cld-blu-03 ~]# grep ENA /etc/sysconfig/yum-autoupdate # ENABLED ENABLED="false" * At least 20GB HD for operating system and OpenStack software and related log files * Dedicated remote storage mounted on ''/var/lib/glance/images'' (which must be already created) where to store the machines images; note that this storage must be shared between the two controller nodes in order to make both glance instances working correctly. This applies to both controller nodes. * SELinux configured as "Disabled" (''/etc/selinux/config'') * EPEL 7 repository configured * A MySQL (possibly a HA cluster) endpoint each OpenStack service can connect to (in this guide we're using our MySQL Percona cluster's IP ''192.168.60.180'') * A HAProxy/Keepalived cluster to use for load-balancing and Virtual IP (in this guide we're using the IP ''192.168.60.180'' for mgmt net and ''90.147.143.10'' for public net) * In the three nodes running HAProxy and in the controller nodes, the following certificates must be installed with the right permissions: * service certificate and key: [root@cld-blu-05 ~]# ll /etc/grid-security/ total 8 -rw-r--r-- 1 root root 1476 May 6 16:59 hostcert.pem -rw------- 1 root root 916 May 6 16:59 hostkey.pem * CA chain certificate: [root@cld-blu-03 ~]# ll /etc/grid-security/chain.pem -rw-r--r--. 1 root root 1257 Mar 24 04:17 /etc/grid-security/chain.pem ===== Configure firewalld to allow traffic through relevant TCP ports on both nodes ===== Execute the following commands on both controller nodes (cld-blu-03 and cld-blu-04): # allow traffic toward rabbitmq server firewall-cmd --permanent --add-port=5672/tcp firewall-cmd --permanent --add-port=4369/tcp firewall-cmd --permanent --add-port=35197/tcp firewall-cmd --permanent --add-port=9100-9110/tcp # allow traffic toward keystone firewall-cmd --permanent --add-port=5000/tcp --add-port=35357/tcp # allow traffic to glance-api firewall-cmd --permanent --add-port=9292/tcp # allow traffic to glance-registry firewall-cmd --permanent --add-port=9191/tcp # allow traffic to Nova EC2 API firewall-cmd --permanent --add-port=8773/tcp # allow traffic to Nova API firewall-cmd --permanent --add-port=8774/tcp # allow traffic to Nova Metadata server firewall-cmd --permanent --add-port=8775/tcp # allow traffic to Nova VNC proxy firewall-cmd --permanent --add-port=6080/tcp # allow traffic to Neutron Server firewall-cmd --permanent --add-port=9696/tcp # allow traffic to Dashboard firewall-cmd --permanent --add-port=80/tcp --add-port=443/tcp # allow traffic to memcached firewall-cmd --permanent --add-port=11211/tcp # allow traffic to Cinder API firewall-cmd --permanent --add-port=3260/tcp --add-port=8776/tcp # permit ntpd's udp communications firewall-cmd --permanent --add-port=123/udp firewall-cmd --reload **WARNING: ''firewall-cmd --reload'' is a destructive command** in regard to any temporary added rules (i.e. those added without ''--permanent'').\\ It is used here only because: - we are configuring firewall from scratch. - rules added with the ''permanent'' directive are not immediately active In the subsequent configurations rules are added by opening ports with the pair * ''firewall-cmd --add-port'' * ''firewall-cmd --permanent --add-port'' Also the HAProxy nodes (cld-blu-05, cld-blu-06 and cld-blu-07) must allow traffic through the same TCP ports: while read i do firewall-cmd --add-port=${i}/tcp firewall-cmd --permanent --add-port=${i}/tcp done << EOF 5672 4369 35197 9100-9110 5000 35357 9292 9191 8773 8774 8775 6080 8776 9696 80 12111 443 8080 8004 8000 EOF firewall-cmd --add-port=123/udp firewall-cmd --permanent --add-port=123/udp ===== Configure HAProxy ===== The HAProxy nodes run the haproxy and keepalived daemons. HAProxy redirects connection from the external world to the controller nodes (users who want to connect to glance/nova/neutron/etc.). Keepalived is responsible to group the HAProxy nodes (3 in our infrastructure) in order to make them highly available with a Virtual Public IP (VIP). This guide will assume (as mentioned above) that HAProxy has been already configured for the MySQL cluster. Only the additional part for OpenStack is shown here. Log into the HAProxy node(s) and put the following lines in ''/etc/haproxy/haproxy.cfg'': global log 127.0.0.1 local0 log 127.0.0.1 local1 notice maxconn 4096 uid 188 gid 188 daemon tune.ssl.default-dh-param 4096 tune.maxrewrite 65536 tune.bufsize 65536 defaults log global mode http option tcplog option dontlognull retries 3 option redispatch maxconn 8000 timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout check 10s listen mysql-cluster-one bind 192.168.60.180:3306 mode tcp balance leastconn option httpchk default-server on-marked-down shutdown-sessions on-marked-up shutdown-backup-sessions server cld-blu-08.cloud.pd.infn.it 192.168.60.157:3306 check port 9200 inter 12000 rise 3 fall 3 server cld-blu-09.cloud.pd.infn.it 192.168.60.158:3306 check port 9200 inter 12000 rise 3 fall 3 backup server cld-blu-10.cloud.pd.infn.it 192.168.60.159:3306 check port 9200 inter 12000 rise 3 fall 3 backup listen mysql-cluster-two bind 192.168.60.180:4306 mode tcp balance leastconn option httpchk default-server on-marked-down shutdown-sessions on-marked-up shutdown-backup-sessions server cld-blu-08.cloud.pd.infn.it 192.168.60.157:3306 check port 9200 inter 12000 rise 3 fall 3 backup server cld-blu-09.cloud.pd.infn.it 192.168.60.158:3306 check port 9200 inter 12000 rise 3 fall 3 server cld-blu-10.cloud.pd.infn.it 192.168.60.159:3306 check port 9200 inter 12000 rise 3 fall 3 backup listen mysql-cluster-three bind 192.168.60.180:5306 mode tcp balance leastconn option httpchk default-server on-marked-down shutdown-sessions on-marked-up shutdown-backup-sessions server cld-blu-08.cloud.pd.infn.it 192.168.60.157:3306 check port 9200 inter 12000 rise 3 fall 3 backup server cld-blu-09.cloud.pd.infn.it 192.168.60.158:3306 check port 9200 inter 12000 rise 3 fall 3 backup server cld-blu-10.cloud.pd.infn.it 192.168.60.159:3306 check port 9200 inter 12000 rise 3 fall 3 listen dashboard_public_ssl bind 90.147.143.10:443 balance source option tcpka option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:443 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:443 check inter 2000 rise 2 fall 3 listen dashboard_public bind 90.147.143.10:80 balance source option tcpka option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:80 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:80 check inter 2000 rise 2 fall 3 listen vnc bind 192.168.60.180:6080 balance source option tcpka option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:6080 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:6080 check inter 2000 rise 2 fall 3 listen vnc_public bind 90.147.143.10:6080 balance source option tcpka option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:6080 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:6080 check inter 2000 rise 2 fall 3 listen keystone_auth_public bind 90.147.143.10:35357 balance source option tcpka option httpchk option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:35357 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:35357 check inter 2000 rise 2 fall 3 listen keystone_api_public bind 90.147.143.10:5000 balance source option tcpka option httpchk option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:5000 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:5000 check inter 2000 rise 2 fall 3 listen keystone_auth bind 192.168.60.180:35357 balance source option tcpka option httpchk option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:35357 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:35357 check inter 2000 rise 2 fall 3 listen keystone_api bind 192.168.60.180:5000 balance source option tcpka option httpchk option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:5000 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:5000 check inter 2000 rise 2 fall 3 listen glance_api bind 192.168.60.180:9292 balance source option tcpka option httpchk option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:9292 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:9292 check inter 2000 rise 2 fall 3 listen glance_api_public bind 90.147.143.10:9292 balance source option tcpka option httpchk option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:9292 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:9292 check inter 2000 rise 2 fall 3 listen glance_registry bind 192.168.60.180:9191 balance source option tcpka option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:9191 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:9191 check inter 2000 rise 2 fall 3 listen novaec2-api bind 192.168.60.180:8773 balance source option tcpka option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:8773 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:8773 check inter 2000 rise 2 fall 3 listen novaec2-api_public bind 90.147.143.10:8773 balance source option tcpka option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:8773 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:8773 check inter 2000 rise 2 fall 3 listen nova-api bind 192.168.60.180:8774 balance source option tcpka option httpchk option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:8774 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:8774 check inter 2000 rise 2 fall 3 listen nova-api_public bind 90.147.143.10:8774 balance source option tcpka option httpchk option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:8774 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:8774 check inter 2000 rise 2 fall 3 listen nova-metadata bind 192.168.60.180:8775 balance source option tcpka option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:8775 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:8775 check inter 2000 rise 2 fall 3 listen nova-metadata_public bind 90.147.143.10:8775 balance source option tcpka option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:8775 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:8775 check inter 2000 rise 2 fall 3 listen cinder-api_public bind 90.147.143.10:8776 balance source option tcpka option tcplog option httpchk server cld-blu-03.cloud.pd.infn.it 192.168.60.152:8776 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:8776 check inter 2000 rise 2 fall 3 listen neutron-server bind 192.168.60.180:9696 balance source option tcpka option httpchk option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:9696 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:9696 check inter 2000 rise 2 fall 3 listen neutron-server_public bind 90.147.143.10:9696 balance source option tcpka option httpchk option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:9696 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:9696 check inter 2000 rise 2 fall 3 listen rabbitmq-server bind 192.168.60.180:5672 balance roundrobin mode tcp server cld-blu-03.cloud.pd.infn.it 192.168.60.152:5672 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:5672 check inter 2000 rise 2 fall 3 listen epmd bind 192.168.60.180:4369 balance roundrobin server cld-blu-03.cloud.pd.infn.it 192.168.60.152:4369 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:4369 check inter 2000 rise 2 fall 3 listen memcached_cluster bind 192.168.60.180:11211 balance source option tcpka option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:11211 check inter 2000 rise 2 fall 5 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:11211 check inter 2000 rise 2 fall 5 Check the syntax of the file you've just modifed: [root@cld-blu-05 ~]# haproxy -c -f /etc/haproxy/haproxy.cfg Configuration file is valid To enable logging of the haproxy traffic allow the rsyslog service on cld-blu-05, cld-blu-06 and cld-blu-07 to accept udp connections from the haproxy daemons. In ''/etc/rsyslog.conf'' uncomment and modify as follows: # Provides UDP syslog reception $ModLoad imudp $UDPServerAddress 127.0.0.1 $UDPServerRun 514 Then add the specific haproxy targets by issuing the command: cat << EOF >> /etc/rsyslog.d/haproxy.conf # Save haproxy messages also to haproxy.log local0.*;local1.* /var/log/haproxy.log EOF Setup logrotate for haproxy.log: cat << EOF >> /etc/logrotate.d/haproxy compress /var/log/haproxy.log { weekly rotate 4 missingok compress minsize 100k } EOF Restart HAProxy and syslog: systemctl restart rsyslog systemctl restart haproxy ===== Create database's tables and users for the Openstack services ===== Login into the MySQL node. Remove previously created users and databases, if any: mysql -u root drop database if exists keystone; drop database if exists glance; drop database if exists nova; drop database if exists neutron; drop database if exists cinder; /* Following commands will raise errors if users are nonexistent. 'drop user if exists' is not implemented in MySQL. http://bugs.mysql.com/bug.php?id=19166 */ drop user 'keystone'@'localhost'; drop user 'keystone'@'192.168.60.%'; drop user 'glance'@'localhost'; drop user 'glance'@'192.168.60.%'; drop user 'nova'@'localhost'; drop user 'nova'@'192.168.60.%'; drop user 'neutron'@'localhost'; drop user 'neutron'@'192.168.60.%'; drop user 'cinder'@'192.168.60.%'; drop user 'cinder'@'localhost'; flush privileges; quit Create database and grant users: mysql -u root CREATE DATABASE keystone; GRANT ALL ON keystone.* TO 'keystone'@'192.168.60.%' IDENTIFIED BY ''; GRANT ALL ON keystone.* TO 'keystone'@'localhost' IDENTIFIED BY ''; CREATE DATABASE glance; GRANT ALL ON glance.* TO 'glance'@'192.168.60.%' IDENTIFIED BY ''; GRANT ALL ON glance.* TO 'glance'@'localhost' IDENTIFIED BY ''; CREATE DATABASE nova; GRANT ALL ON nova.* TO 'nova'@'192.168.60.%' IDENTIFIED BY ''; GRANT ALL ON nova.* TO 'nova'@'localhost' IDENTIFIED BY ''; CREATE DATABASE neutron; GRANT ALL ON neutron.* TO 'neutron'@'192.168.60.%' IDENTIFIED BY ''; GRANT ALL ON neutron.* TO 'neutron'@'localhost' IDENTIFIED BY ''; CREATE DATABASE cinder; GRANT ALL ON cinder.* TO 'cinder'@'192.168.60.%' IDENTIFIED BY ''; GRANT ALL ON cinder.* TO 'cinder'@'localhost' IDENTIFIED BY ''; FLUSH PRIVILEGES; commit; quit Logout from MySQL node. ===== Install OpenStack software on both controller nodes ===== First install the YUM repo from RDO: yum install -y https://repos.fedorapeople.org/repos/openstack/EOL/openstack-icehouse/rdo-release-icehouse-4.noarch.rpm Install the packages for Keystone, Glance, Nova, Neutron and Horizon(Dashboard): yum install -y openstack-keystone python-keystoneclient openstack-utils \ openstack-nova python-novaclient rabbitmq-server openstack-glance \ python-kombu python-anyjson python-amqplib openstack-neutron \ python-neutron python-neutronclient openstack-neutron-openvswitch mariadb \ memcached python-memcached mod_wsgi openstack-dashboard \ openstack-cinder openstack-utils mod_ssl openstack-neutron-ml2 ===== Configure Keystone on primary node ===== Apply a workaround to a known bug (see this [[http://wiki.infn.it/progetti/cloud-areapd/best_practices/optimize_mysql_db_wan_keystone|page]] for more info): openstack-config --set /etc/keystone/keystone.conf token expiration 32400 Proceed with Keystone setup: export SERVICE_TOKEN=$(openssl rand -hex 10) echo $SERVICE_TOKEN > ~/ks_admin_token openstack-config --set /etc/keystone/keystone.conf DEFAULT admin_token $SERVICE_TOKEN openstack-config --set /etc/keystone/keystone.conf sql connection "mysql://keystone:@192.168.60.180:3306/keystone" openstack-config --set /etc/keystone/keystone.conf DEFAULT bind_host 0.0.0.0 keystone-manage pki_setup --keystone-user keystone --keystone-group keystone chown -R keystone:keystone /var/log/keystone /etc/keystone/ssl/ su keystone -s /bin/sh -c "keystone-manage db_sync" Start Keystone: systemctl start openstack-keystone systemctl enable openstack-keystone Get access to Keystone and create the admin user and tenant: export OS_SERVICE_TOKEN=`cat ~/ks_admin_token` export OS_SERVICE_ENDPOINT=http://192.168.60.180:35357/v2.0 keystone service-create --name=keystone --type=identity --description="Keystone Identity Service" The system will respond something like: +-------------+----------------------------------+ | Property | Value | +-------------+----------------------------------+ | description | Keystone Identity Service | | enabled | True | | id | 5363ecce39614aefa80ce8c2f9404691 | | name | keystone | | type | identity | +-------------+----------------------------------+ Subsequent output from the system are suppressed. keystone endpoint-create --service keystone --publicurl http://90.147.143.10:5000/v2.0 --adminurl http://90.147.143.10:35357/v2.0 --internalurl http://192.168.60.180:5000/v2.0 keystone user-create --name admin --pass ADMIN_PASS keystone role-create --name admin keystone tenant-create --name admin keystone role-create --name Member keystone user-role-add --user admin --role admin --tenant admin \rm -f $HOME/keystone_admin.sh echo "export OS_USERNAME=admin" > $HOME/keystone_admin.sh echo "export OS_TENANT_NAME=admin" >> $HOME/keystone_admin.sh echo "export OS_PASSWORD=ADMIN_PASS" >> $HOME/keystone_admin.sh echo "export OS_AUTH_URL=http://90.147.143.10:5000/v2.0/" >> $HOME/keystone_admin.sh keystone tenant-create --name services --description "Services Tenant" ==== Check it ==== In order to check that the Keystone service is well installed, copy the ''keystone_admin.sh'' script you've just created to another machine, even your desktop. Install on it the Python Keystone's command line (''yum -y install python-keystoneclient''); then source the script ''keystone_admin.sh'' and try the command: $ keystone user-list +----------------------------------+-------+---------+-------+ | id | name | enabled | email | +----------------------------------+-------+---------+-------+ | 60aa8974cf4d4736b28b04ffa52492ab | admin | True | | +----------------------------------+-------+---------+-------+ ==== Setup recurring token flush ==== It's better to do this on both controller nodes. See origin of the problem [[http://wiki.infn.it/progetti/cloud-areapd/best_practices/optimize_mysql_db_wan_keystone|here]]. Create the file ''/usr/local/bin/keystone_token_flush.sh'': cat << EOF >> /usr/local/bin/keystone_token_flush.sh #!/bin/sh logger -t keystone-cleaner "Starting token cleanup" /usr/bin/keystone-manage -v -d token_flush logger -t keystone-cleaner "Ending token cleanup" EOF Since the openstack-keystone package rotates all logs in ''/var/log/keystone/'' there is no need to configure the logrotate process any further. Execute: cat << EOF > /etc/cron.hourly/keystone_token_flush /usr/local/bin/keystone_token_flush.sh >> /var/log/keystone/keystone_token_flush.log 2>&1 EOF chmod +x /usr/local/bin/keystone_token_flush.sh chmod 0644 /etc/cron.d/keystone_token_flush ===== Configure RabbitMQ message broker on primary node ===== Define the TCP port range allowed for inter-node communication (this is needed for cluster mode of RabbitMQ) \rm -f /etc/rabbitmq/rabbitmq.config cat << EOF >> /etc/rabbitmq/rabbitmq.config [{kernel, [ {inet_dist_listen_min, 9100}, {inet_dist_listen_max, 9110} ]}]. EOF Correct the logrotate configuration file to use rabbitmqctl instead of the deprecated 'service' command syntax sed -i '/rabbitmq-server rotate-logs/s+/sbin/service rabbitmq-server rotate-logs+/usr/sbin/rabbitmqctl rotate_logs+' /etc/logrotate.d/rabbitmq-server Start and enable Rabbit systemctl start rabbitmq-server systemctl enable rabbitmq-server ===== Configure Glance on primary node ===== Login into the primary controller node, or wherever you've installed the Keystone's command line, and source the script ''keystone_admin.sh'' that you created above: source keystone_admin.sh export OS_SERVICE_TOKEN=`cat ~/ks_admin_token` export OS_SERVICE_ENDPOINT=http://192.168.60.180:35357/v2.0 Ensure SERVICE_ENDPOINT and SERVICE_TOKEN are both set.\\ Then create the Glance user and image service in the Keystone's database: keystone user-create --name glance --pass GLANCE_PASS keystone user-role-add --user glance --role admin --tenant services keystone service-create --name glance --type image --description "Glance Image Service" keystone endpoint-create --service glance --publicurl "http://90.147.143.10:9292" --adminurl "http://90.147.143.10:9292" --internalurl "http://192.168.60.180:9292" Login into the primary controller node, modify the relevant configuration files: **glance-api.conf** while read i do openstack-config --set /etc/glance/glance-api.conf ${i} done << EOF DEFAULT bind_host 0.0.0.0 DEFAULT registry_host 192.168.60.180 DEFAULT notification_driver noop DEFAULT sql_connection "mysql://glance:@192.168.60.180:5306/glance" DEFAULT sql_idle_timeout 30 keystone_authtoken auth_host 192.168.60.180 keystone_authtoken auth_port 35357 keystone_authtoken auth_protocol http keystone_authtoken auth_uri http://192.168.60.180:35357/v2.0 keystone_authtoken admin_tenant_name services keystone_authtoken admin_user glance keystone_authtoken admin_password GLANCE_PASS paste_deploy flavor "keystone+cachemanagement" EOF # The following parameter should equals the CPU number openstack-config --set /etc/glance/glance-api.conf DEFAULT workers 8 **glance-registry.conf** while read i do openstack-config --set /etc/glance/glance-registry.conf ${i} done << EOF DEFAULT bind_host 0.0.0.0 keystone_authtoken admin_tenant_name services keystone_authtoken admin_user glance keystone_authtoken admin_password GLANCE_PASS keystone_authtoken auth_host 192.168.60.180 keystone_authtoken auth_port 35357 keystone_authtoken auth_protocol http keystone_authtoken auth_uri http://192.168.60.180:35357/v2.0 database connection "mysql://glance:@192.168.60.180:5306/glance" database idle_timeout 30 paste_deploy flavor keystone EOF While still logged into the primary controller node, prepare the paths: mkdir -p /var/run/glance /var/log/glance chown -R glance /var/log/glance chown -R glance /var/run/glance chown -R glance:glance /var/lib/glance ... and initialize the Glance's database: # su glance -s /bin/sh /root $ glance-manage db_sync /root $ exit As of 26/02/2015 there is a bug that might prevent the script from working. Check ''/var/log/glance/api.log'' for messages like these: CRITICAL glance [-] ValueError: Tables "migrate_version" have non utf8 collation, please make sure all tables are CHARSET=utf8 glance Traceback (most recent call last): glance File "/usr/bin/glance-manage", line 10, in glance sys.exit(main()) glance File "/usr/lib/python2.7/site-packages/glance/cmd/manage.py", line 259, in main glance return CONF.command.action_fn() glance File "/usr/lib/python2.7/site-packages/glance/cmd/manage.py", line 160, in sync glance CONF.command.current_version) glance File "/usr/lib/python2.7/site-packages/glance/cmd/manage.py", line 137, in sync glance sanity_check=self._need_sanity_check()) glance File "/usr/lib/python2.7/site-packages/glance/openstack/common/db/sqlalchemy/migration.py", line 195, in db_sync glance _db_schema_sanity_check(engine) glance File "/usr/lib/python2.7/site-packages/glance/openstack/common/db/sqlalchemy/migration.py", line 221, in _db_schema_sanity_check glance ) % ','.join(table_names)) glance ValueError: Tables "migrate_version" have non utf8 collation, please make sure all tables are CHARSET=utf8 glance The workaround is (see https://bugs.launchpad.net/oslo-incubator/+bug/1301036/comments/17 ): mysql -u glance -h 192.168.60.10 -p use glance; alter table migrate_version convert to character set utf8; exit; Than reissue # su glance -s /bin/sh /root $ glance-manage db_sync /root $ exit To prevent unprivileged users to register public image, change the policy in ''/etc/glance/policy.json'': "publicize_image": "", to "publicize_image": "role:admin", Always sitting on the primary controller node, start and enable the Glance services: systemctl start openstack-glance-registry systemctl start openstack-glance-api systemctl enable openstack-glance-registry systemctl enable openstack-glance-api ... and finally create the credential file for glance cat << EOF > glancerc export OS_USERNAME=glance export OS_TENANT_NAME=services export OS_PASSWORD=GLANCE_PASS export OS_AUTH_URL=http://192.168.60.180:35357/v2.0/ EOF You can copy the credential file to any machine you like, where you've installed the Python Glance's command line (''yum -y install python-glanceclient''). From this machine you can access the Glance service (list images, create images, delete images, etc.). ==== Check it ==== In order to check that Glance is correctly installed, login into any machines where you've installed the Glance's command line and source the ''glancerc'' script that you've copied from the primary controller node; then try these commands: pcmazzon ~ $ wget http://cdn.download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-disk.img [...] Saving to: “cirros-0.3.1-x86_64-disk.img” [...] 2013-12-06 12:25:03 (3.41 MB/s) - “cirros-0.3.1-x86_64-disk.img” saved [13147648/13147648] pcmazzon ~ $ glance image-create --name=cirros --disk-format=qcow2 --container-format=bare --is-public=True < cirros-0.3.1-x86_64-disk.img +------------------+--------------------------------------+ | Property | Value | +------------------+--------------------------------------+ | checksum | d972013792949d0d3ba628fbe8685bce | | container_format | bare | | created_at | 2015-02-26T16:14:40 | | deleted | False | | deleted_at | None | | disk_format | qcow2 | | id | 0fb09e44-a25c-49e2-a046-191a7989aebc | | is_public | True | | min_disk | 0 | | min_ram | 0 | | name | cirros | | owner | 1af77118d9db4c9a959810aa7d67c6d8 | | protected | False | | size | 13147648 | | status | active | | updated_at | 2015-02-26T16:14:43 | | virtual_size | None | +------------------+--------------------------------------+ pcmazzon ~ $ glance index ID Name Disk Format Container Format Size ------------------------------------ ------------------------------ -------------------- -------------------- -------------- 0fb09e44-a25c-49e2-a046-191a7989aebc cirros qcow2 bare 13147648 ===== Configure Nova on primary node ===== Login into the primary controller node, or wherever you've installed the Keystone's command line, and source the script keystone_admin.sh that you created above: source keystone_admin.sh Add NOVA service, user and endpoint to Keystone's database: keystone user-create --name nova --pass NOVA_PASS keystone user-role-add --user nova --role admin --tenant services keystone service-create --name nova --type compute --description "OpenStack Compute Service" SERVICE_NOVA_ID=`keystone service-list|grep nova|awk '{print $2}'` keystone endpoint-create --service-id $SERVICE_NOVA_ID \ --publicurl http://90.147.143.10:8774/v2/%\(tenant_id\)s \ --adminurl http://90.147.143.10:8774/v2/%\(tenant_id\)s \ --internalurl http://192.168.60.180:8774/v2/%\(tenant_id\)s keystone service-create --name nova_ec2 --type ec2 --description "EC2 Service" SERVICE_EC2_ID=`keystone service-list|grep nova_ec2|awk '{print $2}'` keystone endpoint-create --service-id $SERVICE_EC2_ID \ --publicurl http://90.147.143.10:8773/services/Cloud \ --adminurl http://90.147.143.10:8773/services/Admin \ --internalurl http://192.168.60.180:8773/services/Cloud Login into the primary controller node and modify the relevant configuration files: **nova.conf:** while read i do openstack-config --set /etc/nova/nova.conf ${i} done << EOF database connection "mysql://nova:@192.168.60.180:5306/nova" database idle_timeout 30 DEFAULT rpc_backend nova.openstack.common.rpc.impl_kombu DEFAULT rabbit_hosts 192.168.60.152:5672,192.168.60.153:5672 DEFAULT rabbit_ha_queues True DEFAULT glance_host 192.168.60.180 DEFAULT my_ip 192.168.60.180 DEFAULT vncserver_listen 90.147.143.10 DEFAULT vncserver_proxyclient_address 192.168.60.180 DEFAULT auth_strategy keystone keystone_authtoken auth_host 192.168.60.180 keystone_authtoken auth_protocol http keystone_authtoken auth_port 35357 keystone_authtoken admin_user nova keystone_authtoken admin_tenant_name services keystone_authtoken admin_password NOVA_PASS DEFAULT api_paste_config /etc/nova/api-paste.ini DEFAULT neutron_metadata_proxy_shared_secret METADATA_PASS DEFAULT service_neutron_metadata_proxy true DEFAULT memcached_servers 192.168.60.152:11211,192.168.60.153:11211 DEFAULT enabled_apis ec2,osapi_compute,metadata DEFAULT ec2_listen 0.0.0.0 DEFAULT ec2_listen_port 8773 DEFAULT cpu_allocation_ratio 4.0 EOF # eliminato ????' # openstack-config --set /etc/nova/nova.conf DEFAULT libvirt_vif_driver nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver **api-paste.ini:** while read i do openstack-config --set /etc/nova/api-paste.ini ${i} done < While still logged into the primary controller node, initialize the database (NOTE that this is **db sync** without '_'): # su nova -s /bin/sh /root $ nova-manage db sync /root $ exit Modify the file /etc/nova/policy.json so that users can manage only their VMs: # cd /etc/nova/ # patch -p0 << EOP --- /etc/nova/policy.json.orig 2015-03-04 10:23:54.042132305 +0100 +++ /etc/nova/policy.json 2015-03-04 10:37:32.581084403 +0100 @@ -1,7 +1,8 @@ { "context_is_admin": "role:admin", "admin_or_owner": "is_admin:True or project_id:%(project_id)s", - "default": "rule:admin_or_owner", + "admin_or_user": "is_admin:True or user_id:%(user_id)s", + "default": "rule:admin_or_user", "cells_scheduler_filter:TargetCellFilter": "is_admin:True", @@ -9,6 +10,7 @@ "compute:create:attach_network": "", "compute:create:attach_volume": "", "compute:create:forced_host": "is_admin:True", + "compute:get": "rule:admin_or_owner", "compute:get_all": "", "compute:get_all_tenants": "", "compute:start": "rule:admin_or_owner", EOP You should receive the message: patching file policy.json Start and enable the nova services: systemctl start openstack-nova-api systemctl start openstack-nova-cert systemctl start openstack-nova-consoleauth systemctl start openstack-nova-scheduler systemctl start openstack-nova-conductor systemctl start openstack-nova-novncproxy systemctl enable openstack-nova-api systemctl enable openstack-nova-cert systemctl enable openstack-nova-consoleauth systemctl enable openstack-nova-scheduler systemctl enable openstack-nova-conductor systemctl enable openstack-nova-novncproxy ==== Check it ==== Preferrably from your desktop, or wherever you've copied the ''keystone_admin.sh'' and installed the NOVA's command line, try to execute: pcmazzon ~ $ nova service-list +------------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+ | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | +------------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+ | nova-consoleauth | cld-blu-03.cloud.pd.infn.it | internal | enabled | up | 2015-03-04T09:43:08.000000 | - | | nova-conductor | cld-blu-03.cloud.pd.infn.it | internal | enabled | up | 2015-03-04T09:43:08.000000 | - | | nova-scheduler | cld-blu-03.cloud.pd.infn.it | internal | enabled | up | 2015-03-04T09:43:08.000000 | - | | nova-cert | cld-blu-03.cloud.pd.infn.it | internal | enabled | up | 2015-03-04T09:43:08.000000 | - | +------------------+-----------------------------+----------+---------+-------+----------------------------+-----------------+ pcmazzon ~ $ nova availability-zone-list +--------------------------------+----------------------------------------+ | Name | Status | +--------------------------------+----------------------------------------+ | internal | available | | |- cld-blu-03.cloud.pd.infn.it | | | | |- nova-conductor | enabled :-) 2015-03-04T09:45:18.000000 | | | |- nova-consoleauth | enabled :-) 2015-03-04T09:45:18.000000 | | | |- nova-scheduler | enabled :-) 2015-03-04T09:45:18.000000 | | | |- nova-cert | enabled :-) 2015-03-04T09:45:18.000000 | +--------------------------------+----------------------------------------+ pcmazzon ~ $ nova endpoints +-------------+----------------------------------+ | glance | Value | +-------------+----------------------------------+ | adminURL | http://90.147.143.10:9292 | | id | 3d9b63cc4b624220a3db1a2da99b241f | | internalURL | http://192.168.60.180:9292 | | publicURL | http://90.147.143.10:9292 | | region | regionOne | +-------------+----------------------------------+ +-------------+----------------------------------------------------------------+ | nova | Value | +-------------+----------------------------------------------------------------+ | adminURL | http://90.147.143.10:8774/v2/7114e3c97144459a9c8afbb9f09b4def | | id | 04fcb15180a34ec7a239888decfd55dd | | internalURL | http://192.168.60.180:8774/v2/7114e3c97144459a9c8afbb9f09b4def | | publicURL | http://90.147.143.10:8774/v2/7114e3c97144459a9c8afbb9f09b4def | | region | regionOne | | serviceName | nova | +-------------+----------------------------------------------------------------+ +-------------+-------------------------------------------+ | nova_ec2 | Value | +-------------+-------------------------------------------+ | adminURL | http://90.147.143.10:8773/services/Admin | | id | 36d3d9f4007a4aeabb639530f4400d89 | | internalURL | http://192.168.60.180:8773/services/Cloud | | publicURL | http://90.147.143.10:8773/services/Cloud | | region | regionOne | +-------------+-------------------------------------------+ +-------------+----------------------------------+ | keystone | Value | +-------------+----------------------------------+ | adminURL | http://90.147.143.10:35357/v2.0 | | id | 208f9156abf945509993babdb46579d9 | | internalURL | http://192.168.60.180:5000/v2.0 | | publicURL | http://90.147.143.10:5000/v2.0 | | region | regionOne | +-------------+----------------------------------+ Even better if the above commands can be tried from your desktop, after sourcing the ''keystone_admin.sh''. ==== Create nova user's keypair and distribute them to other nodes ==== usermod -s /bin/bash nova mkdir -p -m 700 ~nova/.ssh chown nova.nova ~nova/.ssh su - nova $ cd .ssh $ ssh-keygen -f id_rsa -b 1024 -P "" $ cp id_rsa.pub authorized_keys $ cat << EOF >> config Host * StrictHostKeyChecking no UserKnownHostsFile=/dev/null EOF $ exit Distribute the content of ''/nova/.ssh'' to the second controller node. ===== Configure Neutron on primary node ===== Login into the primary controller node, or wherever you've installed the Keystone's command line, and source the script keystone_admin.sh that you created above: source ~/keystone_admin.sh Then, create the endpoint, service and user information in the Keystone's database for Neutron: keystone user-create --name neutron --pass NEUTRON_PASS keystone user-role-add --user neutron --role admin --tenant services keystone service-create --name neutron --type network --description "OpenStack Networking Service" SERVICE_NEUTRON_ID=`keystone service-list|grep neutron|awk '{print $2}'` keystone endpoint-create --service-id $SERVICE_NEUTRON_ID \ --publicurl "http://90.147.143.10:9696" \ --adminurl "http://90.147.143.10:9696" \ --internalurl "http://192.168.60.180:9696" Login into the primary controller node and modify the configuration files. **neutron.conf:** while read i do openstack-config --set /etc/neutron/neutron.conf ${i} done << EOF keystone_authtoken auth_host 192.168.60.180 keystone_authtoken admin_tenant_name services keystone_authtoken admin_user neutron keystone_authtoken admin_password NEUTRON_PASS keystone_authtoken auth_url http://192.168.60.180:35357/v2.0 keystone_authtoken auth_uri http://192.168.60.180:35357/v2.0 DEFAULT auth_strategy keystone DEFAULT rpc_backend neutron.openstack.common.rpc.impl_kombu DEFAULT rabbit_hosts 192.168.60.152:5672,192.168.60.153:5672 DEFAULT rabbit_ha_queues True DEFAULT core_plugin ml2 DEFAULT service_plugins router database connection "mysql://neutron:@192.168.60.180:4306/neutron" DEFAULT verbose False DEFAULT dhcp_agents_per_network 2 DEFAULT dhcp_lease_duration 86400 DEFAULT agent_down_time 75 DEFAULT notify_nova_on_port_status_changes True DEFAULT notify_nova_on_port_data_changes True DEFAULT nova_url http://192.168.60.180:8774/v2 DEFAULT nova_admin_username nova #attention the value in the result of command DEFAULT nova_admin_tenant_id $(keystone tenant-list | awk '/ service / { print $2 }') DEFAULT nova_admin_password NOVA_PASS DEFAULT nova_admin_auth_url http://192.168.60.180:35357/v2.0 agent report_interval 30 EOF openstack-config --set /etc/neutron/neutron.conf agent root_helper "sudo neutron-rootwrap /etc/neutron/rootwrap.conf" **api-paste.ini:** while read i do openstack-config --set /etc/neutron/api-paste.ini ${i} done << EOF filter:authtoken paste.filter_factory keystoneclient.middleware.auth_token:filter_factory filter:authtoken admin_tenant_name services filter:authtoken admin_user neutron filter:authtoken admin_password NEUTRON_PASS EOF **ml2_conf.ini:** while read i do openstack-config --set /etc/neutron/plugins/ml2/ml2_conf.ini ${i} done << EOF ml2 type_drivers gre,vlan,flat ml2 tenant_network_types gre ml2 mechanism_drivers openvswitch ml2_type_gre tunnel_id_ranges 1:1000 securitygroup firewall_driver neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver securitygroup enable_security_group True EOF ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini **nova.conf:** while read i do openstack-config --set /etc/nova/nova.conf ${i} done << EOF DEFAULT network_api_class nova.network.neutronv2.api.API DEFAULT neutron_url http://192.168.60.180:9696 DEFAULT neutron_auth_strategy keystone DEFAULT neutron_admin_tenant_name services DEFAULT neutron_admin_username neutron DEFAULT neutron_admin_password NEUTRON_PASS DEFAULT neutron_admin_auth_url http://192.168.60.180:35357/v2.0 DEFAULT linuxnet_interface_driver nova.network.linux_net.LinuxOVSInterfaceDriver DEFAULT firewall_driver nova.virt.firewall.NoopFirewallDriver DEFAULT security_group_api neutron EOF Restart NOVA's services (since you've just modified its configuration file) systemctl restart openstack-nova-api systemctl restart openstack-nova-scheduler systemctl restart openstack-nova-conductor While still logged into the primary controller node, start and enable the Neutron server: neutron-db-manage --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugin.ini stamp head It's output should be like: No handlers could be found for logger "neutron.common.legacy" INFO [alembic.migration] Context impl MySQLImpl. INFO [alembic.migration] Will assume non-transactional DDL. Now start ''neutron-server'' systemctl start neutron-server systemctl enable neutron-server ===== Configure Cinder on primary node ===== Login into the primary controller node, or wherever you've installed the Keystone's command line, and source the script ''keystone_admin.sh'' that you created above: source ~/keystone_admin.sh Then, create the endpoint, service and user information in the Keystone's database for Cinder: keystone user-create --name cinder --pass CINDER_PASS keystone user-role-add --user cinder --role admin --tenant services keystone service-create --name cinder --type volume --description "Cinder Volume Service" keystone service-create --name=cinderv2 --type=volumev2 --description="Cinder Volume Service V2" keystone endpoint-create --service cinder --publicurl http://90.147.143.10:8776/v1/%\(tenant_id\)s --internalurl http://192.168.60.180:8776/v1/%\(tenant_id\)s --adminurl http://90.147.143.10:8776/v1/%\(tenant_id\)s keystone endpoint-create --service cinderv2 --publicurl http://90.147.143.10:8776/v2/%\(tenant_id\)s --internalurl http://192.168.60.180:8776/v2/%\(tenant_id\)s --adminurl http://90.147.143.10:8776/v2/%\(tenant_id\)s Login into the primary controller node and modify the configuration files. **cinder.conf**: while read i do openstack-config --set /etc/cinder/cinder.conf ${i} done << EOF DEFAULT auth_strategy keystone keystone_authtoken auth_host 192.168.60.180 keystone_authtoken admin_tenant_name services keystone_authtoken admin_user cinder keystone_authtoken admin_password CINDER_PASS DEFAULT rpc_backend cinder.openstack.common.rpc.impl_kombu DEFAULT rabbit_hosts 192.168.60.152:5672,192.168.60.153:5672 DEFAULT rabbit_ha_queues True DEFAULT sql_idle_timeout 30 DEFAULT rootwrap_config /etc/cinder/rootwrap.conf DEFAULT api_paste_config /etc/cinder/api-paste.ini DEFAULT control_exchange cinder DEFAULT sql_connection "mysql://cinder:@192.168.60.180:4306/cinder" EOF #openstack-config --set /etc/cinder/cinder.conf DEFAULT osapi_volume_listen 192.168.60.180 Initialize the Cinder database (NOTE that this is **db sync** without '_'): # su cinder -s /bin/sh /root $ cinder-manage db sync /root $ exit Modify the file /etc/cinder/policy.json so that users can manage only their volumes: # cd /etc/cinder/ # patch -p0 << EOP --- policy.json.orig 2015-09-23 10:38:52.499132043 +0200 +++ policy.json 2015-09-22 16:14:20.894780883 +0200 @@ -1,7 +1,8 @@ { "context_is_admin": [["role:admin"]], "admin_or_owner": [["is_admin:True"], ["project_id:%(project_id)s"]], - "default": [["rule:admin_or_owner"]], + "admin_or_user": [["is_admin:True"], ["user_id:%(user_id)s"]], + "default": [["rule:admin_or_user"]], "admin_api": [["is_admin:True"]], EOP You should receive the message: patching file policy.json And finally start API services: systemctl start openstack-cinder-api systemctl enable openstack-cinder-api systemctl start openstack-cinder-scheduler systemctl enable openstack-cinder-scheduler ===== Configure Horizon (Dashboard) on primary node ===== Modify the file ''/etc/openstack-dashboard/local_settings'': look for OPENSTACK_HYPERVISOR_FEATURES = { and set 'can_set_password': True Modify the file ''/etc/openstack-dashboard/local_settings'': look for the CACHES string, and substitute whatever is there with: CACHES = { 'default': { 'BACKEND' : 'django.core.cache.backends.memcached.MemcachedCache', 'LOCATION' : '192.168.60.152:11211', } } you can try this command: sed -i "s+django\.core\.cache\.backends\.locmem\.LocMemCache'+django\.core\.cache\.backends\.memcached\.MemcachedCache',\n\t'LOCATION' : '192.168.60.152:11211',+" /etc/openstack-dashboard/local_settings Note that the TCP port 11211 and IP address must match those ones contained in the file ''/etc/sysconfig/memcached'': PORT="11211" USER="memcached" MAXCONN="1024" CACHESIZE="64" OPTIONS="-l 192.168.60.152" Now, look for the string ''OPENSTACK_HOST'' in /etc/openstack-dashboard/local_settings and set it to: OPENSTACK_HOST = "192.168.60.180" by executing this command: sed -i 's+OPENSTACK_HOST = "127.0.0.1"+OPENSTACK_HOST = "192.168.60.180"+' /etc/openstack-dashboard/local_settings Modify the ''ALLOWED_HOST'' parameter in /etc/openstack-dashboard/local_settings and set it to: ALLOWED_HOSTS = ['*'] by executing the command sed -i "s+ALLOWED_HOSTS = .*+ALLOWED_HOSTS = ['*']+" /etc/openstack-dashboard/local_settings Execute the following commands: sed -i 's+^Listen.*+Listen 192.168.60.152:80+' /etc/httpd/conf/httpd.conf echo "ServerName cld-blu-03.cedc.unipd.it:80" >> /etc/httpd/conf/httpd.conf echo "RedirectMatch permanent ^/$ /dashboard/" >> /etc/httpd/conf.d/openstack-dashboard.conf echo "RedirectMatch ^/$ /dashboard/" > /etc/httpd/conf.d/rootredirect.conf To address an observed problem related to number of open files execute the following command: cat << EOF >> /etc/security/limits.conf * soft nofile 4096 * hard nofile 4096 EOF Start and enable the WebServer: systemctl start httpd systemctl start memcached systemctl enable httpd systemctl enable memcached ==== Configure Dashboard for SSL sessions ==== **Please, do not consider this configuration as optional. It should be done in order to crypt the users' passwords.** Install the ''mod_ssl'' package on both controller nodes: yum -y install mod_ssl Execute the following commands: #sed -i 's+^Listen.*+Listen 8443+' /etc/httpd/conf.d/ssl.conf #sed -i 's+VirtualHost _default_:443+VirtualHost _default_:8443+' /etc/httpd/conf.d/ssl.conf sed -i 's+^SSLCertificateFile.*+SSLCertificateFile /etc/grid-security/hostcert.pem+' /etc/httpd/conf.d/ssl.conf sed -i 's+^SSLCertificateKeyFile.*+SSLCertificateKeyFile /etc/grid-security/hostkey.pem+' /etc/httpd/conf.d/ssl.conf echo "RewriteEngine On" >> /etc/httpd/conf/httpd.conf echo "RewriteCond %{HTTPS} !=on" >> /etc/httpd/conf/httpd.conf echo "RewriteRule ^/?(.*) https://%{SERVER_NAME}:443/\$1 [R,L]" >> /etc/httpd/conf/httpd.conf Restart httpd: systemctl restart httpd ===== ===== ---- **You can stop here if you don't need the High Availability with the second node neither the SSL support.** ---- ===== Configure and "clusterize" RabbitMQ on the secondary node ===== Login into the secondary controller node and configure the RabbitMQ to use the already specified TCP port range: \rm -f /etc/rabbitmq/rabbitmq.config cat << EOF >> /etc/rabbitmq/rabbitmq.config [{kernel, [ {inet_dist_listen_min, 9100}, {inet_dist_listen_max, 9110} ]}]. EOF While still logged into the secondary controller node, start and enable Rabbit: systemctl start rabbitmq-server systemctl enable rabbitmq-server This first start has generated the erlang cookie. Then stop the server: systemctl stop rabbitmq-server RabbitMQ's clustering requires that the nodes have the same Erlang cookie... then copy erlang cookie from the primary node and restart the server: scp root@cld-blu-03.cloud.pd.infn.it:/var/lib/rabbitmq/.erlang.cookie /var/lib/rabbitmq/.erlang.cookie Change cookie's ownership and restart the rabbit server chown rabbitmq:rabbitmq /var/lib/rabbitmq/.erlang.cookie systemctl start rabbitmq-server While logged into the secondary controller node, stop the application: rabbitmqctl stop_app rabbitmqctl reset ... then join the server running in the primary node: rabbitmqctl join_cluster rabbit@cld-blu-03 Clustering node 'rabbit@cld-blu-04' with 'rabbit@cld-blu-03' ... ...done. rabbitmqctl start_app Starting node 'rabbit@cld-blu-04' ... ...done. # see: http://goo.gl/y0aVmp rabbitmqctl set_policy HA '^(?!amq\.).*' '{"ha-mode": "all"}' ==== Check it ==== [root@cld-blu-04 ~]# rabbitmqctl cluster_status Cluster status of node 'rabbit@cld-blu-04' ... [{nodes,[{disc,['rabbit@cld-blu-03','rabbit@cld-blu-04']}]}, {running_nodes,['rabbit@cld-blu-03','rabbit@cld-blu-04']}, {partitions,[]}] ...done. [root@cld-blu-04 ~]# rabbitmqctl list_policies Listing policies ... / HA ^(?!amq\\.).* {"ha-mode":"all"} 0 ...done. [root@cld-blu-03 ~]# rabbitmqctl cluster_status Cluster status of node 'rabbit@cld-blu-03' ... [{nodes,[{disc,['rabbit@cld-blu-03','rabbit@cld-blu-04']}]}, {running_nodes,['rabbit@cld-blu-04','rabbit@cld-blu-03']}, {partitions,[]}] ...done. [root@cld-blu-03 ~]# rabbitmqctl list_policies Listing policies ... / HA ^(?!amq\\.).* {"ha-mode":"all"} 0 ...done. ===== Configure services on secondary node ===== Login into the secondary controller node; copy Keystone, Glance, Nova, Neutron, Cinder and Horizon's configurations from primary controller node: scp cld-blu-03.cloud.pd.infn.it:/etc/openstack-dashboard/local_settings /etc/openstack-dashboard/ scp -r cld-blu-03.cloud.pd.infn.it:/etc/keystone /etc/ scp -r cld-blu-03.cloud.pd.infn.it:/etc/neutron /etc/ scp -r cld-blu-03.cloud.pd.infn.it:/etc/cinder /etc/ scp -r cld-blu-03.cloud.pd.infn.it:/etc/glance /etc/ scp -r cld-blu-03.cloud.pd.infn.it:/etc/nova /etc/ scp cld-blu-03.cloud.pd.infn.it:/etc/sysconfig/memcached /etc/sysconfig/ \rm -f /etc/neutron/plugin.ini ln -s /etc/neutron/plugins/ml2/ml2_conf.ini /etc/neutron/plugin.ini While still logged into the secondary controller node, finalize the setup: keystone-manage pki_setup --keystone-user keystone --keystone-group keystone mkdir -p /var/run/glance /var/log/glance mkdir -p /var/run/keystone /var/log/keystone chown -R glance:glance /var/log/glance /var/lib/glance /var/run/glance chown -R keystone:keystone /var/run/keystone /var/log/keystone /var/lib/keystone /etc/keystone/ssl/ chown -R neutron:neutron /var/lib/neutron Setup Dashboard's parameters: sed -i 's+^Listen.*+Listen 192.168.60.153:80+' /etc/httpd/conf/httpd.conf echo "ServerName cloud.cedc.csia.unipd.it:80" >> /etc/httpd/conf/httpd.conf echo "RedirectMatch permanent ^/$ /dashboard/" >> /etc/httpd/conf.d/openstack-dashboard.conf echo "RedirectMatch ^/$ /dashboard/" > /etc/httpd/conf.d/rootredirect.conf Setup HTTPS for Dashboard: sed -i 's+^SSLCertificateFile.*+SSLCertificateFile /etc/grid-security/hostcert.pem+' /etc/httpd/conf.d/ssl.conf sed -i 's+^SSLCertificateKeyFile.*+SSLCertificateKeyFile /etc/grid-security/hostkey.pem+' /etc/httpd/conf.d/ssl.conf echo "RewriteEngine On" >> /etc/httpd/conf/httpd.conf echo "RewriteCond %{HTTPS} !=on" >> /etc/httpd/conf/httpd.conf echo "RewriteRule ^/?(.*) https://%{SERVER_NAME}:443/\$1 [R,L]" >> /etc/httpd/conf/httpd.conf Increase the number of allowed open files: cat << EOF >> /etc/security/limits.conf * soft nofile 4096 * hard nofile 4096 EOF ... change the memcached's listening IP address sed -i 's+192.168.60.152+192.168.60.153+' /etc/sysconfig/memcached ... change the location of the memcached service in the dashboard's config file: sed -i 's+192.168.60.152:11211+192.168.60.153:11211+' /etc/openstack-dashboard/local_settings ... and finally turn all services ON, and enable them: systemctl start openstack-keystone systemctl start openstack-glance-registry systemctl start openstack-glance-api systemctl start openstack-nova-api systemctl start openstack-nova-cert systemctl start openstack-nova-consoleauth systemctl start openstack-nova-scheduler systemctl start openstack-nova-conductor systemctl start openstack-nova-novncproxy systemctl start neutron-server systemctl start httpd systemctl start memcached systemctl start openstack-cinder-api systemctl start openstack-cinder-scheduler systemctl enable openstack-keystone systemctl enable openstack-glance-registry systemctl enable openstack-glance-api systemctl enable openstack-nova-api systemctl enable openstack-nova-cert systemctl enable openstack-nova-consoleauth systemctl enable openstack-nova-scheduler systemctl enable openstack-nova-conductor systemctl enable openstack-nova-novncproxy systemctl enable neutron-server systemctl enable httpd systemctl enable memcached systemctl enable openstack-cinder-api systemctl enable openstack-cinder-scheduler ==== Check it ==== On your desktop, source the file ''keystone_admin.sh'' and try the commands: bash-4.1$ nova availability-zone-list +--------------------------------+----------------------------------------+ | Name | Status | +--------------------------------+----------------------------------------+ | internal | available | | |- cld-blu-03.cloud.pd.infn.it | | | | |- nova-conductor | enabled :-) 2015-03-10T09:22:08.000000 | | | |- nova-consoleauth | enabled :-) 2015-03-10T09:21:59.000000 | | | |- nova-scheduler | enabled :-) 2015-03-10T09:22:04.000000 | | | |- nova-cert | enabled :-) 2015-03-10T09:22:02.000000 | | |- cld-blu-04.cloud.pd.infn.it | | | | |- nova-conductor | enabled :-) 2015-03-10T09:22:00.000000 | | | |- nova-cert | enabled :-) 2015-03-10T09:22:01.000000 | | | |- nova-consoleauth | enabled :-) 2015-03-10T09:22:04.000000 | | | |- nova-scheduler | enabled :-) 2015-03-10T09:22:00.000000 | +--------------------------------+----------------------------------------+ bash-4.1$ cinder service-list +------------------+-----------------------------+------+---------+-------+----------------------------+ | Binary | Host | Zone | Status | State | Updated_at | +------------------+-----------------------------+------+---------+-------+----------------------------+ | cinder-scheduler | cld-blu-03.cloud.pd.infn.it | nova | enabled | up | 2015-03-10T09:23:09.000000 | | cinder-scheduler | cld-blu-04.cloud.pd.infn.it | nova | enabled | up | 2015-03-10T09:23:05.000000 | +------------------+-----------------------------+------+---------+-------+----------------------------+ ===== Optional: SSL configuration & INFN-AAI ===== First of all, on both controller nodes, switch off all the OpenStack's services **but Keystone** (also do not stop ''**memcached**''): systemctl stop openstack-glance-registry systemctl stop openstack-glance-api systemctl stop openstack-nova-api systemctl stop openstack-nova-cert systemctl stop openstack-nova-consoleauth systemctl stop openstack-nova-scheduler systemctl stop openstack-nova-conductor systemctl stop openstack-nova-novncproxy systemctl stop neutron-server systemctl stop httpd systemctl stop openstack-cinder-api systemctl stop openstack-cinder-scheduler ==== Configure HAProxy to act like an SSL terminator ==== Before proceed note that * **HAProxy 1.5.x** is required to support an SSL frontend. * The ''hostcert.pem'' and ''hostkey.pem'' files must be concatenated (with the ''cat'' command) to create the unique file ''hostcertkey.pem'' Modify ''haproxy.cfg'''s content in the haproxy nodes, by substituting the lines listed [[progetti:cloud-areapd:ced-c:controller_and_network_nodes?&#configure_haproxy|above]] with the following (do not modify the two sections ''global'' and ''defaults''): listen mysql-cluster-roundrobin bind 192.168.60.180:3306 mode tcp balance roundrobin option httpchk server cld-blu-10.cloud.pd.infn.it 192.168.60.159:3306 check port 9200 inter 12000 rise 3 fall 3 server cld-blu-09.cloud.pd.infn.it 192.168.60.158:3306 check port 9200 inter 12000 rise 3 fall 3 server cld-blu-08.cloud.pd.infn.it 192.168.60.157:3306 check port 9200 inter 12000 rise 3 fall 3 listen mysql-cluster-roundrobin-public bind 90.147.143.10:3306 mode tcp balance roundrobin option httpchk server cld-blu-10.cloud.pd.infn.it 192.168.60.159:3306 check port 9200 inter 12000 rise 3 fall 3 server cld-blu-09.cloud.pd.infn.it 192.168.60.158:3306 check port 9200 inter 12000 rise 3 fall 3 server cld-blu-08.cloud.pd.infn.it 192.168.60.157:3306 check port 9200 inter 12000 rise 3 fall 3 listen nova-metadata bind 192.168.60.180:8775 balance source option tcpka option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:8775 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:8775 check inter 2000 rise 2 fall 3 listen glance-registry bind 192.168.60.180:9191 balance source option tcpka option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:9191 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:9191 check inter 2000 rise 2 fall 3 listen rabbitmq-server bind 192.168.60.180:5672 balance roundrobin mode tcp server cld-blu-03.cloud.pd.infn.it 192.168.60.152:5672 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:5672 check inter 2000 rise 2 fall 3 listen epmd bind 192.168.60.180:4369 balance roundrobin server cld-blu-03.cloud.pd.infn.it 192.168.60.152:4369 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:4369 check inter 2000 rise 2 fall 3 listen memcached-cluster bind 192.168.60.180:11211 balance source option tcpka option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:11211 check inter 2000 rise 2 fall 5 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:11211 check inter 2000 rise 2 fall 5 frontend dashboard-ssl_public bind 90.147.143.10:443 option tcplog mode tcp default_backend dashboard_ssl_nodes frontend keystone-auth bind 192.168.60.180:35357 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend keystoneauth frontend keystone-api bind 192.168.60.180:5000 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend keystoneauth frontend keystone-auth_public bind 90.147.143.10:35357 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend keystoneauth frontend keystone-api_public bind 90.147.143.10:5000 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend keystoneapi frontend glance-api bind 192.168.60.180:9292 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend glanceapi frontend glance-api_public bind 90.147.143.10:9292 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend glanceapi frontend nova-api bind 192.168.60.180:8774 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend novaapi frontend nova-api_public bind 90.147.143.10:8774 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend novaapi frontend cinder-api bind 192.168.60.180:8776 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend cinderapi frontend cinder-api_public bind 90.147.143.10:8776 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend cinderapi frontend neutron-server bind 192.168.60.180:9696 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend neutronapi frontend neutron-server_public bind 90.147.143.10:9696 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend neutronapi frontend vnc_public bind 90.147.143.10:6080 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend novnc frontend novaec2-api bind 192.168.60.180:8773 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend ec2api frontend novaec2-api_public bind 90.147.143.10:8773 ssl crt /etc/grid-security/hostcertkey.pem ca-file /etc/grid-security/chain.pem mode http option httpclose option forwardfor reqadd X-Forwarded-Proto:\ https default_backend ec2api backend dashboard_ssl_nodes balance source mode tcp option ssl-hello-chk server cld-blu-03.cloud.pd.infn.it 192.168.60.152:443 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:443 check inter 2000 rise 2 fall 3 backend keystoneauth mode http balance source option httpchk server cld-blu-03.cloud.pd.infn.it 192.168.60.152:35357 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:35357 check inter 2000 rise 2 fall 3 backend keystoneapi mode http balance source option httpchk server cld-blu-03.cloud.pd.infn.it 192.168.60.152:5000 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:5000 check inter 2000 rise 2 fall 3 backend glanceapi mode http balance source option httpchk server cld-blu-03.cloud.pd.infn.it 192.168.60.152:9292 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:9292 check inter 2000 rise 2 fall 3 backend novaapi mode http balance source option httpchk server cld-blu-03.cloud.pd.infn.it 192.168.60.152:8774 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:8774 check inter 2000 rise 2 fall 3 backend ec2api mode http balance source option tcpka option tcplog server cld-blu-03.cloud.pd.infn.it 192.168.60.152:8773 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:8773 check inter 2000 rise 2 fall 3 backend cinderapi mode http balance source option httpchk server cld-blu-03.cloud.pd.infn.it 192.168.60.152:8776 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:8776 check inter 2000 rise 2 fall 3 backend neutronapi mode http balance source option httpchk server cld-blu-03.cloud.pd.infn.it 192.168.60.152:9696 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:9696 check inter 2000 rise 2 fall 3 backend novnc mode http balance source server cld-blu-03.cloud.pd.infn.it 192.168.60.152:6080 check inter 2000 rise 2 fall 3 server cld-blu-04.cloud.pd.infn.it 192.168.60.153:6080 check inter 2000 rise 2 fall 3 ... and restart the HAProxy daemon. Now login into one of the two controller nodes, and do a precautionary unset all the OS_* variables: unset OS_USERNAME unset OS_TENANT_NAME unset OS_PASSWORD unset OS_AUTH_URL To get back access to Keystone issue the following commands: export SERVICE_TOKEN=`cat ~/ks_admin_token` export SERVICE_ENDPOINT=http://192.168.60.152:35357/v2.0 * Note 1: 192.168.60.152 is the IP address of the controller node you're logged into. * Note 2: the file ''~/ks_admin_token'' has been created above at the very first Keystone's configuration. Change Keystone's endpoints: KEYSTONE_SERVICE=$(keystone service-get keystone | grep ' id ' | awk '{print $4}') KEYSTONE_ENDPOINT=$(keystone endpoint-list | grep $KEYSTONE_SERVICE|awk '{print $2}') keystone endpoint-delete $KEYSTONE_ENDPOINT keystone endpoint-create --region RegionOne --service-id $KEYSTONE_SERVICE --publicurl "https://cloud.cedc.csia.unipd.it:\$(public_port)s/v2.0" --adminurl "https://cloud.cedc.csia.unipd.it:\$(admin_port)s/v2.0" --internalurl "https://cloud.cedc.csia.unipd.it:\$(public_port)s/v2.0" Note: no need to restart ''keystone'' because its communication is still not encrypted; the encription only occurs in the HAProxy frontend. Change the ''keystone_admin.sh'' you created above: sed -i 's+export OS_AUTH_URL+#export OS_AUTH_URL+' $HOME/keystone_admin.sh echo "export OS_AUTH_URL=https://cloud.cedc.csia.unipd.it:5000/v2.0/" >> $HOME/keystone_admin.sh echo "export OS_CACERT=/etc/grid-security/chain.pem" >> $HOME/keystone_admin.sh Note: no need to do anything on the second controller node. The endpoint with the https url has been changed on the MySQL database; all this is transparent for the ''keystone'' service. ==== Check it ==== unset SERVICE_TOKEN unset SERVICE_ENDPOINT source $HOME/keystone_admin.sh keystone user-list ==== Glance ==== Modify authentication parameters on both controller nodes: while read i do openstack-config --set /etc/glance/glance-api.conf ${i} done << EOF keystone_authtoken auth_host cloud.cedc.csia.unipd.it keystone_authtoken auth_protocol https keystone_authtoken auth_uri https://cloud.cedc.csia.unipd.it:35357/v2.0 keystone_authtoken cafile /etc/grid-security/chain.pem EOF while read i do openstack-config --set /etc/glance/glance-registry.conf ${i} done << EOF keystone_authtoken auth_host cloud.cedc.csia.unipd.it keystone_authtoken auth_protocol https keystone_authtoken auth_uri https://cloud.cedc.csia.unipd.it:35357/v2.0 keystone_authtoken cafile /etc/grid-security/chain.pem EOF Execute this on one controller node only (or where you have the file ''keystone_admin.sh''): source ~/keystone_admin.sh GLANCE_SERVICE=$(keystone service-get glance | grep ' id ' | awk '{print $4}') GLANCE_ENDPOINT=$(keystone endpoint-list | grep $GLANCE_SERVICE|awk '{print $2}') keystone endpoint-delete $GLANCE_ENDPOINT keystone endpoint-create --service glance --publicurl "https://cloud.cedc.csia.unipd.it:9292" --adminurl "https://cloud.cedc.csia.unipd.it:9292" --internalurl "https://cloud.cedc.csia.unipd.it:9292" Restart Glance on both controller nodes: systemctl restart openstack-glance-api systemctl restart openstack-glance-registry ==== Nova ==== Modify authentication parameters on both controller nodes: while read i do openstack-config --set /etc/nova/nova.conf ${i} done << EOF keystone_authtoken auth_host cloud.cedc.csia.unipd.it keystone_authtoken auth_protocol https keystone_authtoken cafile /etc/grid-security/chain.pem DEFAULT neutron_ca_certificates_file /etc/grid-security/chain.pem DEFAULT cinder_ca_certificates_file /etc/grid-security/chain.pem DEFAULT glance_host cloud.cedc.csia.unipd.it DEFAULT glance_protocol https DEFAULT glance_api_servers https://cloud.cedc.csia.unipd.it:9292 DEFAULT glance_api_insecure true DEFAULT neutron_url https://cloud.cedc.csia.unipd.it:9696 EOF openstack-config --set /etc/nova/api-paste.ini filter:authtoken auth_host cloud.cedc.csia.unipd.it openstack-config --set /etc/nova/api-paste.ini filter:authtoken auth_protocol https openstack-config --set /etc/nova/api-paste.ini filter:authtoken auth_uri https://cloud.cedc.csia.unipd.it:5000/v2.0 On one controller node only (or where you have ''keystone_admin.sh''): NOVA_SERVICE=$(keystone service-get nova | grep ' id ' | awk '{print $4}') NOVA_ENDPOINT=$(keystone endpoint-list | grep $NOVA_SERVICE|awk '{print $2}') keystone endpoint-delete $NOVA_ENDPOINT keystone endpoint-create --service-id $NOVA_SERVICE --publicurl https://cloud.cedc.csia.unipd.it:8774/v2/%\(tenant_id\)s --adminurl https://cloud.cedc.csia.unipd.it:8774/v2/%\(tenant_id\)s --internalurl https://cloud.cedc.csia.unipd.it:8774/v2/%\(tenant_id\)s NOVAEC2_SERVICE=$(keystone service-get nova_ec2 | grep ' id ' | awk '{print $4}') NOVAEC2_ENDPOINT=$(keystone endpoint-list | grep $NOVAEC2_SERVICE|awk '{print $2}') keystone endpoint-delete $NOVAEC2_ENDPOINT keystone endpoint-create --service-id $NOVAEC2_SERVICE --publicurl https://cloud.cedc.csia.unipd.it:8773/services/Cloud --adminurl https://cloud.cedc.csia.unipd.it:8773/services/Cloud --internalurl https://cloud.cedc.csia.unipd.it:8773/services/Cloud Restart Nova on both controller nodes: systemctl restart openstack-nova-api systemctl restart openstack-nova-cert systemctl restart openstack-nova-consoleauth systemctl restart openstack-nova-scheduler systemctl restart openstack-nova-conductor systemctl restart openstack-nova-novncproxy ==== Neutron ==== Apply the patches in https://review.openstack.org/#/c/90626/ to allow SSL neutron -> nova communications. Modify authentication parameters on both controller nodes: while read i do openstack-config --set /etc/neutron/neutron.conf ${i} done << EOF keystone_authtoken auth_protocol https keystone_authtoken auth_host cloud.cedc.csia.unipd.it keystone_authtoken auth_url https://cloud.cedc.csia.unipd.it:35357/v2.0 keystone_authtoken auth_uri https://cloud.cedc.csia.unipd.it:35357/v2.0 keystone_authtoken cafile /etc/grid-security/chain.pem DEFAULT nova_protocol https DEFAULT nova_url https://cloud.cedc.csia.unipd.it:8774/v2 DEFAULT nova_admin_username nova #DEFAULT nova_admin_tenant_id=$(keystone tenant-list | awk '/ services / { print$2 }') DEFAULT nova_admin_tenant_id 1af77118d9db4c9a959810aa7d67c6d8 DEFAULT nova_admin_password NOVA_PASS DEFAULT nova_admin_auth_url https://cloud.cedc.csia.unipd.it:35357/v2.0 DEFAULT nova_ca_certificates_file /etc/grid-security/chain.pem DEFAULT notify_nova_on_port_status_changes True DEFAULT notify_nova_on_port_data_changes True EOF openstack-config --set /etc/nova/nova.conf DEFAULT neutron_admin_auth_url https://cloud.cedc.csia.unipd.it:35357/v2.0 On one controller only (or where you have the ''keystone_admin.sh''): NEUTRON_SERVICE=$(keystone service-get neutron | grep ' id ' | awk '{print $4}') NEUTRON_ENDPOINT=$(keystone endpoint-list | grep $NEUTRON_SERVICE|awk '{print $2}') keystone endpoint-delete $NEUTRON_ENDPOINT keystone endpoint-create --service-id $NEUTRON_SERVICE --publicurl "https://cloud.cedc.csia.unipd.it:9696" --adminurl "https://cloud.cedc.csia.unipd.it:9696" --internalurl "https://cloud.cedc.csia.unipd.it:9696" Restart Neutron and Nova on both controller nodes (nova needs to be restarted because its conf file has been changed): systemctl restart neutron-server systemctl restart openstack-nova-api systemctl restart openstack-nova-cert systemctl restart openstack-nova-consoleauth systemctl restart openstack-nova-scheduler systemctl restart openstack-nova-conductor systemctl restart openstack-nova-novncproxy ==== Cinder ==== Modify authentication parameters on both controller nodes: while read i do openstack-config --set /etc/cinder/cinder.conf ${i} done << EOF keystone_authtoken auth_host cloud.cedc.csia.unipd.it keystone_authtoken auth_protocol https keystone_authtoken auth_uri https://cloud.cedc.csia.unipd.it:5000/v2.0 keystone_authtoken cafile /etc/grid-security/chain.pem EOF On one controller only (or where you have the ''keystone_admin.sh''): CINDER_SERVICE=$(keystone service-get cinder | grep ' id ' | awk '{print $4}') CINDER_ENDPOINT=$(keystone endpoint-list | grep $CINDER_SERVICE|awk '{print $2}') keystone endpoint-delete $CINDER_ENDPOINT CINDER_SERVICE=$(keystone service-get cinderv2 | grep ' id ' | awk '{print $4}') CINDER_ENDPOINT=$(keystone endpoint-list | grep $CINDER_SERVICE|awk '{print $2}') keystone endpoint-delete $CINDER_ENDPOINT keystone endpoint-create  --service cinder --publicurl https://cloud.cedc.csia.unipd.it:8776/v1/%\(tenant_id\)s --adminurl https://cloud.cedc.csia.unipd.it:8776/v1/%\(tenant_id\)s --internalurl https://cloud.cedc.csia.unipd.it:8776/v1/%\(tenant_id\)s keystone endpoint-create  --service cinderv2 --publicurl https://cloud.cedc.csia.unipd.it:8776/v2/%\(tenant_id\)s --adminurl https://cloud.cedc.csia.unipd.it:8776/v2/%\(tenant_id\)s --internalurl https://cloud.cedc.csia.unipd.it:8776/v2/%\(tenant_id\)s Restart Cinder on both controller nodes: systemctl restart openstack-cinder-api systemctl restart openstack-cinder-scheduler ==== Horizon (both controller nodes) ==== Setup secure connection to Keystone: sed -i 's+OPENSTACK_HOST = "192.168.60.180"+OPENSTACK_HOST = "cloud.cedc.csia.unipd.it"+' /etc/openstack-dashboard/local_settings sed -i 's+OPENSTACK_KEYSTONE_URL = "http:+OPENSTACK_KEYSTONE_URL = "https:+' /etc/openstack-dashboard/local_settings sed -i 's+# OPENSTACK_SSL_CACERT.*+OPENSTACK_SSL_CACERT="/etc/grid-security/chain.pem"+' /etc/openstack-dashboard/local_settings Those two are [[http://docs.openstack.org/developer/horizon/topics/deployment.html#secure-site-recommendations|recommended for SSL]] sed -i 's+#CSRF_COOKIE_SECURE = True+CSRF_COOKIE_SECURE = True+' /etc/openstack-dashboard/local_settings sed -i 's+#SESSION_COOKIE_SECURE = True+SESSION_COOKIE_SECURE = True+' /etc/openstack-dashboard/local_settings Prepare to patch Horizon's source files: # # NOT APPLIED # Is it still needed? # yum install -y patch curl -o os_auth_patch_01.diff https://raw.githubusercontent.com/CloudPadovana/SSL_Patches/master/os_auth_patch_01.diff curl -o os_auth_patch_02.diff https://raw.githubusercontent.com/CloudPadovana/SSL_Patches/master/os_auth_patch_02.diff curl -o os_auth_patch_03.diff https://raw.githubusercontent.com/CloudPadovana/SSL_Patches/master/os_auth_patch_03.diff patch -R /usr/lib/python2.6/site-packages/openstack_auth/views.py < os_auth_patch_01.diff patch -R /usr/lib/python2.6/site-packages/openstack_auth/backend.py < os_auth_patch_02.diff patch -R /usr/lib/python2.6/site-packages/openstack_auth/user.py < os_auth_patch_03.diff Restart apache web server: systemctl restart httpd ==== Integration of INFN-AAI in Keystone (on both controller nodes) ==== See [[https://wiki.infn.it/progetti/_DASHBOARD_WEB_ACCESS_/integration_of_the_infn-aai_authentication_in_the_openstack_keystone|here]] === ===