Some notes for LVM

Unfuck it after second disk gone in restore case

We had a database server, we needed restore from a few days ago.

The LVM had two pv disks, one with snapshots enabled, one without.

We needed to restore /home/exports corresponding to vg_datas-lv_mysql_exports.

This will work because, by luck, /dev/sdb was snapshotted, and /dev/sdc wasn't, AND specially because the lv_mysql_exports LV was on the /dev/sdb disk.

As shown in this snippet from the live server:

[uwu@source-server:~]$ sudo lsblk
NAME                              MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                                 8:0    0    20G  0 disk
├─sda1                              8:1    0   200M  0 part /boot/efi
└─sda2                              8:2    0  19.8G  0 part /
sdb                                 8:16   0   1.2T  0 disk
├─vg_datas-lv_mysql_data          253:0    0 919.7G  0 lvm  /var/lib/mysql/data
├─vg_datas-lv_mysql_binlogs       253:1    0   309G  0 lvm  /var/lib/mysql/binlogs
├─vg_datas-lv_mysql_exports       253:2    0   140G  0 lvm  /home/exports
└─vg_datas-lv_xxx                 253:3    0    10G  0 lvm  /home/applis/xxx
sdc                                 8:32   0   300G  0 disk
├─vg_datas-lv_mysql_data          253:0    0 919.7G  0 lvm  /var/lib/mysql/data
└─vg_datas-lv_xxx                 253:4    0    50G  0 lvm  /var/lib/xxx

We created a new VM, with disks from the available snapshots, and got:

[root@srv-restore ~]# pvs
  WARNING: Device for PV d59f9dc7-3b1b-43a4-a0d1-96a230b324cb not found or rejected by a filter.
  Couldn't find device with uuid d59f9dc7-3b1b-43a4-a0d1-96a230b324cb.
  PV         VG       Fmt  Attr PSize    PFree
  /dev/sdb   vg_datas lvm2 a--     1.19t      0
  [unknown]  vg_datas lvm2 a-m  <300.00g <91.34g

A bit annoying, as we can't get any lv mounted or shown at all.

We are going to fake a disk with the wanted UUID, only to makes LVM happy and get what we want.

Export the UUID from the pvs command:

export UUID="d59f9dc7-3b1b-43a4-a0d1-96a230b324cb"

Create a 100M disk-file:

dd if=/dev/zero of=/tmp/tmp.raw bs=1M count=100

Some losetup magic to map the file to a block device:

losetup -f
losetup /dev/loop0 /tmp/tmp.raw

Create a PV from this "disk":

pvcreate --norestorefile -u $UUID /dev/loop0

And now, LVM is happy !

[root@srv-restore ~]# pvs
  WARNING: Device /dev/loop0 has size of 204800 sectors which is smaller than corresponding PV size of 629145600 sectors. Was device resized?
  One or more devices used as PVs in VG vg_datas have changed sizes.
  PV         VG       Fmt  Attr PSize    PFree
  /dev/loop0 vg_datas lvm2 a--  <300.00g <91.34g
  /dev/sdb   vg_datas lvm2 a--     1.19t      0
[root@srv-restore ~]# lvs
  WARNING: Device /dev/loop0 has size of 204800 sectors which is smaller than corresponding PV size of 629145600 sectors. Was device resized?
  One or more devices used as PVs in VG vg_datas have changed sizes.
  LV                     VG       Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  lv_xxx                 vg_datas -wi-------  50.00g
  lv_mysql_binlogs       vg_datas -wi------- 309.00g
  lv_mysql_data          vg_datas -wi------- 919.65g
  lv_mysql_exports       vg_datas -wi-a----- 140.00g
  lv_xxx                 vg_datas -wi-------  10.00g

We might had needed to do a vgchange -ay or something to get the vg and lvs to shows, but in the end, we were able to ignore the warning, and mount /dev/vg_datas/lv_mysql_exports without issues.

And since srv-restore was temporary, we nuked everything.

If that was on a non-temporary server, you would need to properly remove the LV/VG/PVs, disable losetup with losetup -D, and remove /tmp/tmp.raw.