Monthly Archives: February 2016

Recovering LVM from a failed resize

I was recently doing a disk resize in VMWare and ran into a problem where LVM could no longer find the PV after I had done the fsck, even though I was sure I set the start sector correctly on the new partition I’d created.
It later became apparent that fdisk had been running with units set to “Cylinders” for some¬†reason and put the partition start sector at a slightly different location, which meant LVM could no longer find the metadata for the PV and assumed it was missing.

Luckily I was able to correct this by looking at the partition table of a similar server, but this lead me to work on finding a way to locate the start position should I find myself needing to recover from such an issue again, I eventually came up with the below:

until pvck -vv --labelsector $SECTOR /dev/sda 2>&1 | grep "for sector 1";
done | awk '{sector=$9-$5; if (sector > 0) print "Partition starts at sector: "sector; }'

This will loop through looking for a valid LVM header then if/when it finds one it will output the location of the PVs start sector, you can then use fdisk to create a partition at that location.

If you have some idea of where the partition was, you can set the SECTOR variable to a sector number just before you expect your start sector to be, this should save a bunch of time from searching every sector on the disk up until that point.

To prevent this happening again I generally take a backup of the partition table using sfdisk, i,e

sfdisk -d /dev/sda | tee ~/sda.txt

Copy/Paste the output of this command to notepad or something similar, so that in the event of an issue, you can pipe the text back into sfdisk and restore the previous partition layout i.e

sfdisk /dev/sda < ~/sda.txt