After deleting snapshots on a VM either by deleting an individual snapshot or selecting "Delete All" snapshots, you may see the following warning for the VM, stating that disk consolidation is needed.

"Virtual machine disks consolidation is needed"

 This can occur when a snapshot has been deleted and removed from snapshot manager, but the consolidation of the VMDKs on disk have failed.

  

You can initiate a consolidation of the VMDKs manually by right clicking on the VM and selecting Snapshot --> Consolidate.

However the consolidate operation may fail again, if the issue which caused the snapshot deletion operation to fail disk consolidation previously has not been cleared.

 

It has been a good 6 months or more since I've last seen this issue, but today I found a VM with this issue in another customers environment.

This is what had happened:

  • A single snapshot had been deleted on a VM
  • The disk consolidation needed warning was shown afterwards
  • VM -> Snapshot -> Consolidate fails with locked file error

Initially I tried the following which can sometimes clear this:

  • Shutdown guest OS and power off VM (it's not always possible to do this but I could here)
  • Create a new snapshot
  • "Delete All" snapshots
  • vMotion to another host
  • Try  VM -> Snapshot -> Consolidate

 

But this did not clear the locked file error as shown below:

"Unable to access file since it is locked"

 

Keen to locate the problem I SSH'd to the ESXi server and checked the hostd.log

Even with the VM still powered off, I could see it was having a problem locking one of the VMDKs when tying to run the Snapshot -> Consolidate task.

tail -f /var/log/hostd.log

 

I then ran the following command to locate which host(s) had the lock on the VMDK stated in hostd.log

vmkfstools -D /vmfs/volumes/yourvolume/yourVM/yourlockedVM.vmdk

I could see a single entry for "RO Owner" which had the lock and the MAC address ending in "69a0".

This is the ESXi server which has the lock on the VMDK file.

 

Next locate which ESXi host has a network adaptor with that MAC address.

Once confirmed I placed the host in maintenance mode, DRS vMotioned all VMs to another host in the cluster and restarted the hostd service.

/etc/init.d/hostd restart

Once the hostd service had restarted I performed a Snapshot -> Consolidate on the VM and it completed successfully.

 

This issue can often occur when a virtual machine backup solution creates a lock on a VMDK and fails to correctly release it.

In this case the snapshot which had been deleted, was left over from an automated backup snapshot which had not been removed by the backup solution once it had completed. 

 

Share this blog post on social media:

Social Links

Disclaimer

All advice, installation/configuration how to guides, troubleshooting and other information on this website are provided as-is with no warranty or guarantee. Whilst the information provided is correct to the best of my knowledge, I am not reponsible for any issues that may arise using this information, and you do so at your own risk. As always before performing anything; check, double check, test and always ensure you have a backup.

Copyright ©2016 Andy Barnes - Please do not copy any content including images without prior consent!

Designed and Hosted by Andy Barnes