INFN wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Remove a compute node howto ======

Let's suppose that we want to remove, temporarily (e.g. for some maintenance operations) or forever (e.g. because it must be dismissed) a compute node (e.g. ''cld-nl-02.cloud.pd.infn.it'') from the Cloud production service).
Please take into account also the notes for the [[http://wiki.infn.it/progetti/cloud-areapd/operations/live_migration_of_virtual_machines|Live Migration howto]] 

===== Disable the compute node =====

Disable the compute node so that it is not used anymore to host new instances:

<code bash>
# . keystone_admin.sh
# nova-manage service disable cld-nl-02.cloud.pd.infn.it nova-compute 
</code>


===== Migrate the VMs hosted on that compute node to other compute nodes =====

Find the VMs hosted in the relevant compute node:

<code bash>
# nova list --host cld-nl-02.cloud.pd.infn.it --all-tenants 


+--------------------------------------+-------------+--------+------------+-------------+-----------------------------+
| ID                                   | Name        | Status | Task State | Power State | Networks                    |
+--------------------------------------+-------------+--------+------------+-------------+-----------------------------+
| 61449b0d-d890-457e-8ab5-a4fa29cc622c | mypubvm     | ACTIVE | -          | Running     | AdminTesting-wan=10.63.15.2 |
| 182a276f-7278-4d34-82d8-e8c58630a8b8 | myvm        | ACTIVE | -          | Running     | AdminTesting-lan=10.64.15.9 |
| ec8cad8a-ad38-4f0f-adc6-b79a1c46079d | testandroid | ACTIVE | -          | Running     | Admin-lan=10.64.1.5         |
+--------------------------------------+-------------+--------+------------+-------------+-----------------------------+
</code>

Migrate them to other compute node(s) (''cld-nl-03.lnl.infn.it'') in the following example:

<code bash>
# nova live-migration 61449b0d-d890-457e-8ab5-a4fa29cc622c cld-nl-03.cloud.pd.infn.it
# nova live-migration 182a276f-7278-4d34-82d8-e8c58630a8b8 cld-nl-03.cloud.pd.infn.it
# nova live-migration ec8cad8a-ad38-4f0f-adc6-b79a1c46079d cld-nl-03.cloud.pd.infn.it
</code>

Check that indeed there are no other instances on the relevant compute node

<code bash>
# nova list --host cld-nl-02.cloud.pd.infn.it --all-tenants +----+------+--------+------------+-------------+----------+
| ID | Name | Status | Task State | Power State | Networks |
+----+------+--------+------------+-------------+----------+
+----+------+--------+------------+-------------+----------+
</code>

Now the compute node can be shutdown

===== Remove persistently the compute node =====

*Warning: the following operations haven't been fully tested*

If and only if the compute node must be removed forever (e.g. because it has to be dismissed):

   * remove its network agents

<code bash>

</code>
   * remove it from the OpenStack Nova database:

<code bash>
mysql> use nova_prod;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed

mysql> delete from compute_node_stats where compute_node_id=(select cn.id from compute_nodes cn, services s where cn.service_id=s.id and s.host='cld-nl-02.cloud.pd.infn.it');
Query OK, 0 rows affected (0.00 sec)

mysql> delete from compute_nodes where service_id=(select id from services where host='cld-nl-02.cloud.pd.infn.it');
Query OK, 1 row affected (0.01 sec)

mysql> delete from services where host='cld-nl-02.cloud.pd.infn.it';
Query OK, 1 row affected (0.01 sec)
</code>

   * remove the endpoints of the relevant compute nodes from the OpenStack Neutron  database (in the following example the compute node to be removed has IP 192.168.61.59):

<code bash>
mysql> use neutron_prod;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from ovs_tunnel_endpoints; 
+----------------+----+
| ip_address     | id |
+----------------+----+
| 192.168.61.105 |  1 |
| 192.168.61.106 |  2 |
| 192.168.61.59  |  4 |
| 192.168.61.114 |  5 |
| 192.168.61.115 |  6 |
| 192.168.61.57  |  7 |
| 192.168.61.58  |  8 |
+----------------+----+
6 rows in set (0.00 sec)
mysql> delete from ovs_tunnel_endpoints where id = 4;
</code>

   * ''service neutron-openvswitch-agent restart'' on all nodes