VMWARE: Datastore fails to unmount due to a locked file

When trying to unmount a datastore in VMWARE an error occurs due to a file still existing on the datastore. The file will not delete in the GUI or from the cli.

1. Discovering a file is lock
# rm VDI-Desktop-028-vdm-disposable-4f70ddb4-6699-4649-85e0-d1fce33f6e0a.vmdk
rm: can’t remove ‘VDI-Desktop-028-vdm-disposable-4f70ddb4-6699-4649-85e0-d1fce33f6e0a.vmdk’: Device or resource busy

2. Look to see what has the file locked
# lsof | grep -i VDI-Desktop-028-vdm-disposable-4f70ddb4-6699-4649-85e0-d1fce33f6e0a.vmdk
/vmfs/volumes/51e5bda0-d77e7dbd-c71c-00215e5d489c/VDI-Desktop-028

3. In my case, there is nothing showing up as having the file locked

4. Now determine the state of the file with vmkfstools. For the purposes of this write-up we will cd into the needed directory.

# vmkfstools -D VDI-Desktop-028-vdm-disposable-4f70ddb4-6699-4649-85e0-d1fce33f6e0a.vmdk

Lock [type 10c00001 offset 92305408 v 246409, hb offset 4186112
gen 857, mode 2, owner 00000000-00000000-0000-000000000000 mtime 148164482
num 1 gblnum 0 gblgen 0 gblbrk 0]
RO Owner[0] HB Offset 3842048 587d159e-9ab8bf0c-687b-0025b5001b29
Addr <4, 196, 79>, gen 246048, links 1, type reg, flags 0, uid 0, gid 0, mode 600
len 576, nb 0 tbz 0, cow 0, newSinceEpoch 0, zla 4305, bs 8192

5.Refer to the above UUID of the system that has it locked: 0025b5001b29

6. You can look for the UUID of each system by logging onto the with the CLI or use VCENTER

CLI
# esxcfg-info -y | grep “System UUID”
|—-System UUID………………………………………….5880b297-eadd-89ba-c3c4-0025b5001c0a

Web GUI
Go to: Host and Clusters – Choose an ESX Host – Networking – Physical Adapters – MAC Address

7. Once you have determined which ESX host has locked the file you will need to reboot the server.
8. Assuming that your server is in a cluster with HA and DRS running, put the ESX server in maintenance mode
9. Once the virtual machines have vacated your server right click on the server and reboot.
10. At this point you can log onto another ESX host in the same cluster and delete your previously locked file.

# rm VDI-Desktop-028-vdm-disposable-4f70ddb4-6699-4649-85e0-d1fce33f6e0a.vmdk -f
/vmfs/volumes/51e5bda0-d77e7dbd-c71c-00215e5d489c/VDI-Desktop-028

11. After your server comes back up exit maintenance mode
12. You can now unmount your data store.

If you were examining th file from a different ESX server, be certain to cd out of the volume on the datastore in order to be able to delete the datastore from Vcenter

By: Timothy Conrad