Table of Contents
Installation and configuration of Ceilometer on IceHouse
HAProxy configuration
On each HAproxy server, add the following entry in /etc/haproxy/haproxy.cfg:
listen ceilometer-api-public bind 192.168.60.40:8777 balance source option tcpka option tcplog server cld-ctrl-01.cloud.pd.infn.it 192.168.60.105:8777 check inter 2000 rise 2 fall 3 server cld-ctrl-02.cloud.pd.infn.it 192.168.60.106:8777 check inter 2000 rise 2 fall 3
and then restart the haproxy process:
service haproxy restart
Installation and Configuration of clusterize MongoDB (Replicaset)
We assume to have 3 database nodes on which to install and clusterize MongoDB.
Software Install & Configuration (all database nodes)
yum -y install mongodb mongodb-server
Configure mongo to listen on management IP (make sure the $MYIP
env var contains what you expect before using it):
sed -i 's,^bind_ip,#bind_ip,' /etc/mongodb.conf MYIP=`hostname -i` cat << EOF >> /etc/mongodb.conf smallfiles = true bind_ip = $MYIP EOF
Now set the env var $DBPATH
to the actual path you want the database stores its files into:
export DBPATH=/var/lib/<SOME_MOUNTED_LARGE_FS>/mongodb
Now configure mongo to use $DBPATH
:
sed -i 's,^dbpath,#dbpath,' /etc/mongodb.conf cat << EOF >> /etc/mongodb.conf dbpath = $DBPATH EOF
Now configure mongo to use replica set:
sed -i 's,^replSet,#replSet,' /etc/mongodb.conf cat << EOF >> /etc/mongodb.conf replSet = rs0 EOF
Now start and enable the service:
service mongod start; chkconfig mongod on
Create the replica cluster (on one database node only)
Connect to the local mongo service with the "mongo
" command line:
mongo --host `hostname -i`
While inside the mongo shell execute the following commands to initiate and create the three-nodes cluster:
rs.initiate() rs.add('<X1.Y1.Z1>') rs.add('<X2.Y2.Z2>') rs.conf() { "_id" : "rs0", "version" : 3, "members" : [ { "_id" : 0, "host" : "<X0.Y0.Z0>:27017" }, { "_id" : 1, "host" : "<X1.Y1.Z1>:27017" }, { "_id" : 2, "host" : "<X2.Y2.Z2>:27017" } ] }
where X0.Y0.Z0
is the output of "hostname -i
" on current node, X1.Y1.Z1 and X2.Y2.Z2 are the output of "hostname -i
" on the other two database nodes.
Now check the cluster status:
rs.status() { "set" : "rs0", "date" : ISODate("2015-08-03T10:05:16Z"), "myState" : 1, "members" : [ { "_id" : 0, "name" : "<X0.Y0.Z0>:27017", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 339180, "optime" : Timestamp(1438596213, 400), "optimeDate" : ISODate("2015-08-03T10:03:33Z"), "self" : true }, { "_id" : 1, "name" : "<X1.Y1.Z1>:27017", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 339132, "optime" : Timestamp(1438596213, 400), "optimeDate" : ISODate("2015-08-03T10:03:33Z"), "lastHeartbeat" : ISODate("2015-08-03T10:05:14Z"), "lastHeartbeatRecv" : ISODate("2015-08-03T10:05:14Z"), "pingMs" : 0, "syncingTo" : "<X0.Y0.Z0>:27017" }, { "_id" : 2, "name" : "<X2.Y2.Z2>:27017", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 339128, "optime" : Timestamp(1438596213, 400), "optimeDate" : ISODate("2015-08-03T10:03:33Z"), "lastHeartbeat" : ISODate("2015-08-03T10:05:14Z"), "lastHeartbeatRecv" : ISODate("2015-08-03T10:05:14Z"), "pingMs" : 0, "syncingTo" : "<X0.Y0.Z0>:27017" } ], "ok" : 1 }
Check that "syncingTo
" parameters have the value of the "PRIMARY
" (the node where rs.initiate()
was executed).
Create the Ceilometer database
Log into one of the mongodb nodes and issue the command:
mongo --eval ' db = db.getSiblingDB("ceilometer"); db.addUser({user: "ceilometer", pwd: "<CEILOMETER_DB_PWD>", roles: [ "readWrite", "dbAdmin" ]})'
Installation and configuration of first controller node
yum install openstack-ceilometer-api openstack-ceilometer-collector \
openstack-ceilometer-notification openstack-ceilometer-central openstack-ceilometer-alarm \
python-ceilometerclient
MONGO_CLUSTER=xxx:27017,yyy:27017,zzz:27017 openstack-config --set /etc/ceilometer/ceilometer.conf database connection "mongodb://ceilometer:<CEILOMETER_DB_PWD>@$MONGO_CLUSTER/ceilometer"
CEILOMETER_TOKEN=$(openssl rand -hex 10) openstack-config --set /etc/ceilometer/ceilometer.conf publisher metering_secret $CEILOMETER_TOKEN openstack-config --set /etc/ceilometer/ceilometer.conf DEFAULT rpc_backend ceilometer.openstack.common.rpc.impl_kombu openstack-config --set /etc/ceilometer/ceilometer.conf DEFAULT rabbit_hosts 192.168.60.11:5672,192.168.60.12:5672,192.168.60.13:5672 openstack-config --set /etc/ceilometer/ceilometer.conf DEFAULT rabbit_ha_queues true
keystone user-create --name=ceilometer --pass=CEILOMETER_PASS --email=ceilometer@example.com keystone user-role-add --user=ceilometer --tenant=services --role=admin
openstack-config --set /etc/ceilometer/ceilometer.conf DEFAULT auth_strategy keystone openstack-config --set /etc/ceilometer/ceilometer.conf database time_to_live 2764800 openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken auth_host cloud-areapd.pd.infn.it openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken admin_user ceilometer openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken admin_tenant_name services openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken auth_protocol https openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken auth_uri https://cloud-areapd.pd.infn.it:5000 openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken admin_password CEILOMETER_PASS openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken cafile /etc/grid-security/certificates/INFN-CA-2006.pem openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_auth_url https://cloud-areapd.pd.infn.it:5000/v2.0 openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_username ceilometer openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_tenant_name services openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_password CEILOMETER_PASS openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_cacert /etc/grid-security/certificates/INFN-CA-2006.pem
keystone service-create --name=ceilometer --type=metering --description="Telemetry" keystone endpoint-create --service-id=$(keystone service-list | awk '/ metering / {print $2}') \ --publicurl=http://$CONTROLLER_VIP_MGMT:8777 \ --internalurl=http://$CONTROLLER_VIP_MGMT:8777 \ --adminurl=http://$CONTROLLER_VIP_MGMT:8777
service openstack-ceilometer-api start service openstack-ceilometer-notification start service openstack-ceilometer-central start service openstack-ceilometer-collector start service openstack-ceilometer-alarm-evaluator start service openstack-ceilometer-alarm-notifier start chkconfig openstack-ceilometer-api on chkconfig openstack-ceilometer-notification on chkconfig openstack-ceilometer-central on chkconfig openstack-ceilometer-collector on chkconfig openstack-ceilometer-alarm-evaluator on chkconfig openstack-ceilometer-alarm-notifier on
Glance in telemetry:
openstack-config --set /etc/glance/glance-api.conf DEFAULT notification_driver messaging openstack-config --set /etc/glance/glance-api.conf DEFAULT rpc_backend glance.openstack.common.rpc.impl_kombu openstack-config --set /etc/glance/glance-api.conf DEFAULT rabbit_hosts 192.168.60.11:5672,192.168.60.12:5672,192.168.60.13:5672 service openstack-glance-api restart service openstack-glance-registry restart
Cinder in telemetry:
openstack-config --set /etc/cinder/cinder.conf DEFAULT control_exchange cinder openstack-config --set /etc/cinder/cinder.conf DEFAULT notification_driver cinder.openstack.common.notifier.rpc_notifier service openstack-cinder-api restart service openstack-cinder-scheduler restart service openstack-cinder-volume restart
Installation and configuration of second controller node
yum install openstack-ceilometer-api openstack-ceilometer-collector \
openstack-ceilometer-notification openstack-ceilometer-central openstack-ceilometer-alarm \
python-ceilometerclient
scp -r cld-ctrl-01:/etc/ceilometer /etc
service openstack-ceilometer-api start service openstack-ceilometer-notification start service openstack-ceilometer-central start service openstack-ceilometer-collector start service openstack-ceilometer-alarm-evaluator start service openstack-ceilometer-alarm-notifier start chkconfig openstack-ceilometer-api on chkconfig openstack-ceilometer-notification on chkconfig openstack-ceilometer-central on chkconfig openstack-ceilometer-collector on chkconfig openstack-ceilometer-alarm-evaluator on chkconfig openstack-ceilometer-alarm-notifier on
Installation and configuration on compute nodes
yum -y install openstack-ceilometer-compute python-ceilometerclient python-pecan
openstack-config --set /etc/nova/nova.conf DEFAULT instance_usage_audit True openstack-config --set /etc/nova/nova.conf DEFAULT instance_usage_audit_period hour openstack-config --set /etc/nova/nova.conf DEFAULT notify_on_state_change vm_and_task_state
Edit the /etc/nova/nova.conf file and add the following lines to the [DEFAULT] section:

[DEFAULT] ... notification_driver = nova.openstack.common.notifier.rpc_notifier notification_driver = ceilometer.compute.nova_notifier
Restart the Compute service:
systemctl restart openstack-nova-compute
CEIOLOMETER_TOKEN=<quello generato nel controller node> openstack-config --set /etc/ceilometer/ceilometer.conf publisher metering_secret $CEILOMETER_TOKEN openstack-config --set /etc/ceilometer/ceilometer.conf DEFAULT rpc_backend ceilometer.openstack.common.rpc.impl_kombu openstack-config --set /etc/ceilometer/ceilometer.conf DEFAULT rabbit_hosts 192.168.60.11:5672,192.168.60.12:5672,192.168.60.13:5672 openstack-config --set /etc/ceilometer/ceilometer.conf DEFAULT rabbit_ha_queues true openstack-config --set /etc/ceilometer/ceilometer.conf DEFAULT auth_strategy keystone openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken auth_host cloud-areapd.pd.infn.it openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken admin_user ceilometer openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken admin_tenant_name services openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken auth_protocol https openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken admin_password CEILOMETER_PASS openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken auth_uri https://cloud-areapd.pd.infn.it:5000 openstack-config --set /etc/ceilometer/ceilometer.conf keystone_authtoken cafile /etc/grid-security/certificates/INFN-CA-2006.pem openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_username ceilometer openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_tenant_name services openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_password CEILOMETER_PASS openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_auth_url https://cloud-areapd-test.pd.infn.it:5000/v2.0 openstack-config --set /etc/ceilometer/ceilometer.conf service_credentials os_cacert /etc/grid-security/certificates/INFN-CA-2006.pem
systemctl start openstack-ceilometer-compute
systemctl enable openstack-ceilometer-compute