I wanted to blog here about a silly mistake I made yesterday.
I need to expand a VM’s disk and was using the ol’ vmkfstools -X. However, I obviously do this all too flippantly and forgot to check if there was any snapshots. oops. That sunken feeling.
I corrupted the vmdk and the snapshot vmsd file.
Here’s how I fixed it all.
Firstly I couldn’t run anything against the original vmdk file, as it kept saying the file was locked by the host. I unregistered the VM from VC, put the ESX server into maintenance mode (which moved all the VMs off to other hosts in the clustering – fully enabled DRS) and power cycled the host. This remove the lock on the file thankfully.
The original vmdk had been 20GB before I tried expanding it to 30GB. So I created a new 20GB vmdk:
# vmkfstools -c 20G -d thick silver1.vmdk
I then copied the contents of the broken 30GB disk into the new one:
# dd if=silver-flat.vmdk of=silver1-flat.vmdk bs=1048576 count=20480
Moved the old vmdk and renamed the new one:
# vmkfstools -E silver.vmdk silver.vmdk.old
# vmkfstools -E silver1.vmdk silver.vmdk
All the snapshots rely upon a CID and a parentCID in their -00000x.vmdk files. I opened both snapshot disks to record the details.
# cat silver-000001.vmdk
# cat silver-000002.vmdk
Basically, 000002 parentCID is 000001’s CID. In turn 000001 parentID was set to point to the original vmdk. So I had to edit the new vmdk file and set this CID.
# vi silver.vmdk
I was then able to power on the VM. Phew!
However the snapshots were still screwed up. When I tries to commit the snapshots with:
# vmware-cmd silver.vmx removesnapshots
It thought it didn’t have any snapshots, but it was clearly using the two additional vmdk disks. The vmsd file was also corrupted. So I renamed the existing one:
# mv silver.vmsd silver.vmsd.old
Then I created a new snapshot, to re-generate a new vmsd file:
# vmware-cmd silver.vmx createsnapshot recreate_vmsd
And finally I could commit the snapshots:
# vmware-cmd silver.vmx removesnapshots
Now all that was left was to increase the disk:
#vmkfstools -X 30G silver.vmdk
I guess I won’t be doing that in a hurry again 🙂
So if I have a problem with vcenter having a snapshot in snapshot manager (when there isnt one cause I had to clone it – the options was greyed out on or off) I could just recreate the .VMSD file and it should all be fine?
Hi Chad,
Personally, I would try to create a new snapshot then delete it at the CLI first. This should stop it thinking there are any previous snapshots.
If you get really stuck, you can afford a few minutes downtime, and you don’t need any snapshots in the clone, you could:
– shut it off
– unregister it in vCenter
– create a new VM in vCenter in a different datastore (or if you only have 1 datastore, then rename the old folder first)
– move the VMDK base files from the old folder to the new folder from the vCenter GUI (this copies the flat file and descriptor file)
– edit the new VM’s settings to add the existing disks now in its folder
Hope that helps.
You understand therefore significantly in the case of this subject, made me personally believe it from a lot of varied angles. Its like men and women aren’t interested unless it’s one thing to do with Woman gaga! Your individual stuffs nice. At all times maintain it up!
We use ESXi 5 U2 – simply powering off VM, remove from inventory, readd to inventory resolved this issue for us