Thursday, January 15, 2015

How to delete and detach iSCSI volumes manually in a ESXi Shell

In this article we will look how to detach and delete iSCSI volumes manually from ESXi.
This article will focus in how to on ESXi Shell only.

We had some issues with some hosts before. Because of wrong deleted or detaching iSCSI volumes and not a good cleaning.

Note: Before we start, some VMware official information and technical details you can get in the VMware KB 1014953


Migrate or backup any data you should have inside of the Volumes you want to delete/detach.

For this tasks and for the safe side, we recommend do this tasks in maintenance mode, even is not mandatory.

Connect the host(where we want to delete/detach the volumes) and use ESXi Shell.

For deleting or detaching Volumes(disk and LUNs) we need to collect disk and LUN information from within ESXi(naa, iqn and Datastore names and numbers, etc.)

First we will collect naa id using the command: esxcli storage vmfs extent list

We will remove the LUN(Datastore) called Storage-001 and clean all iSCSI connections etc.

This is the information I get and that we will use for this article:

~ # esxcli storage vmfs extent list
Volume Name  VMFS UUID                            Extent Number  Device Name                           Partition
-----------  -----------------------------------  -------------  ------------------------------------  ---------
Storage-001  54b7c326-6204faef-6719-e839351060ad              0  naa.6090a06860d40e5a524b15ae2bc3c366          1


In this case our Volume is Storage-001 and the naa id is naa.6090a06860d40e5a524b15ae2bc3c366

1. Unmount the Volume(file system) from the host:
# esxcli storage filesystem unmount -l Storage-001

After this you can connect to your Storage Administration and put the Volume offline. If you put your Volume offline, you can bypass the step 3 and move to step 4.

3. Disable the iSCSI Device(using naa id):
# esxcli storage core device set --state=off -d naa.6090a06860d40e5a524b15ae2bc3c366

Next we need to know which iSCSI adapter we are using and also which iSCSI server(sendtarget) is in the Static Discover so that we can remove it also(only if we need to do a full cleanup and remove also the Storage from the VMware environment).

~ # esxcli iscsi adapter discovery sendtarget list
Adapter  Sendtarget
-------  ---------------
vmhba37  10.10.10.33:3260


Note before go to step 4: Do this step only if you want to delete completely this Storage from your VMware environment. If you have more than one Volumes from this Storage, if this is the case skip this step and move to step 5.

4. Remove the discovery address from the Static Discovery in the iSCSI Initiator:
# esxcli iscsi adapter discovery sendtarget remove --adapter=vmhba37 --address='10.10.10.33:3260'

Next we need to check which iSCSI sessions there is in the target list toward this volumes that we want to detach.

~ # esxcli iscsi adapter target portal list
Adapter  Target                                                                      IP          Port  Tpgt
-------  --------------------------------------------------------------------------  ----------  ----  ----
vmhba37  iqn.2001-05.com.equallogic:0-8a0906-5a0ed4606-66c3c32bae154b52-storage-001  10.10.10.33  3260  1


This is the target adapter, now list the sessions(since the Best Pratices for iSCSI connections is to have 2 portgroups for High Availability and multipath purposes we will see two sessions for each Volume).
We just need to remove the target. In our case is: iqn.2001-05.com.equallogic:0-8a0906-5a0ed4606-66c3c32bae154b52-storage-001

Note: The next print have more details, but we will only list the firsts lines(the more detail information is not needed for our tasks)

~ # esxcli iscsi session list
vmhba37,iqn.2001-05.com.equallogic:0-8a0906-5a0ed4606-66c3c32bae154b52-storage-001,00023d000001
   Adapter: vmhba37
   Target: iqn.2001-05.com.equallogic:0-8a0906-5a0ed4606-66c3c32bae154b52-storage-001
   ISID: 00023d000001

vmhba37,iqn.2001-05.com.equallogic:0-8a0906-5a0ed4606-66c3c32bae154b52-storage-001,00023d000002
   Adapter: vmhba37
   Target: iqn.2001-05.com.equallogic:0-8a0906-5a0ed4606-66c3c32bae154b52-storage-001
   ISID: 00023d000002


5. Remove iSCSI target session:
# esxcli iscsi session remove -A vmhba37 -n iqn.2001-05.com.equallogic:0-8a0906-5a0ed4606-66c3c32bae154b52-storage-001

6. Next check if the target was removed and rescan the adapter:
# esxcli storage core adapter rescan --adapter=vmhba37
# esxcli iscsi adapter target portal list and # esxcli iscsi session list

You should see no output after the command.

Now for the final cleanup, we need to check the detached device list and remove any entries for the volume that we removed/detached using: esxcli storage core device detached list

~ # esxcli storage core device detached list
Device UID                            State
------------------------------------  -----
naa.6090a06860d40e5a524b15ae2bc3c366  off


If is empty, you can bypass this step and finish.

7. Check and clean the device detached list:
# esxcli storage core device detached remove -d naa.6090a06860d40e5a524b15ae2bc3c366

Check the device detached list again and see if is empty

Note: You can use some of this commands that we present in this article to to clean some old detached Volumes that are still around in your VMware environment from a non good cleaning.

After this you can now delete the Volume in your Storage.

Hope this can can help you fixing this issues.

PS: I will like t thank you my colleague Marc Therman for the help testing some of this stuff.