User Tools

Site Tools


progetti:cloud-areapd:operations:production_cloud:remove_a_compute_node_howto

Remove a compute node howto

Let's suppose that we want to remove, temporarily (e.g. for some maintenance operations) or forever (e.g. because it must be dismissed) a compute node (e.g. cld-nl-02.cloud.pd.infn.it) from the Cloud production service). Please take into account also the notes for the Live Migration howto

Disable the compute node

Disable the compute node so that it is not used anymore to host new instances:

# . keystone_admin.sh
# nova-manage service disable cld-nl-02.cloud.pd.infn.it nova-compute 

Migrate the VMs hosted on that compute node to other compute nodes

Find the VMs hosted in the relevant compute node:

# nova list --host cld-nl-02.cloud.pd.infn.it --all-tenants 
 
 
+--------------------------------------+-------------+--------+------------+-------------+-----------------------------+
| ID                                   | Name        | Status | Task State | Power State | Networks                    |
+--------------------------------------+-------------+--------+------------+-------------+-----------------------------+
| 61449b0d-d890-457e-8ab5-a4fa29cc622c | mypubvm     | ACTIVE | -          | Running     | AdminTesting-wan=10.63.15.2 |
| 182a276f-7278-4d34-82d8-e8c58630a8b8 | myvm        | ACTIVE | -          | Running     | AdminTesting-lan=10.64.15.9 |
| ec8cad8a-ad38-4f0f-adc6-b79a1c46079d | testandroid | ACTIVE | -          | Running     | Admin-lan=10.64.1.5         |
+--------------------------------------+-------------+--------+------------+-------------+-----------------------------+

Migrate them to other compute node(s) (cld-nl-03.lnl.infn.it) in the following example:

# nova live-migration 61449b0d-d890-457e-8ab5-a4fa29cc622c cld-nl-03.cloud.pd.infn.it
# nova live-migration 182a276f-7278-4d34-82d8-e8c58630a8b8 cld-nl-03.cloud.pd.infn.it
# nova live-migration ec8cad8a-ad38-4f0f-adc6-b79a1c46079d cld-nl-03.cloud.pd.infn.it

Check that indeed there are no other instances on the relevant compute node

# nova list --host cld-nl-02.cloud.pd.infn.it --all-tenants +----+------+--------+------------+-------------+----------+
| ID | Name | Status | Task State | Power State | Networks |
+----+------+--------+------------+-------------+----------+
+----+------+--------+------------+-------------+----------+

Now the compute node can be shutdown

Remove persistently the compute node

*Warning: the following operations haven't been fully tested*

If and only if the compute node must be removed forever (e.g. because it has to be dismissed):

  • remove its network agents
 
  • remove it from the OpenStack Nova database:
mysql> use nova_prod;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
 
Database changed
 
mysql> delete from compute_node_stats where compute_node_id=(select cn.id from compute_nodes cn, services s where cn.service_id=s.id and s.host='cld-nl-02.cloud.pd.infn.it');
Query OK, 0 rows affected (0.00 sec)
 
mysql> delete from compute_nodes where service_id=(select id from services where host='cld-nl-02.cloud.pd.infn.it');
Query OK, 1 row affected (0.01 sec)
 
mysql> delete from services where host='cld-nl-02.cloud.pd.infn.it';
Query OK, 1 row affected (0.01 sec)
  • remove the endpoints of the relevant compute nodes from the OpenStack Neutron database (in the following example the compute node to be removed has IP 192.168.61.59):
mysql> use neutron_prod;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
 
Database changed
mysql> select * from ovs_tunnel_endpoints; 
+----------------+----+
| ip_address     | id |
+----------------+----+
| 192.168.61.105 |  1 |
| 192.168.61.106 |  2 |
| 192.168.61.59  |  4 |
| 192.168.61.114 |  5 |
| 192.168.61.115 |  6 |
| 192.168.61.57  |  7 |
| 192.168.61.58  |  8 |
+----------------+----+
6 rows in set (0.00 sec)
mysql> delete from ovs_tunnel_endpoints where id = 4;
  • service neutron-openvswitch-agent restart on all nodes
progetti/cloud-areapd/operations/production_cloud/remove_a_compute_node_howto.txt · Last modified: 2015/04/22 13:49 by fantinel@infn.it

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki