LVM Snapshot Merge Fails After lvconvert --merge

Why Your LVM Snapshot Merge Failed

This always happens when you run lvconvert --merge on a snapshot, but something goes sideways mid-merge. The culprit is almost always a filesystem still mounted on the origin volume, a stale lock, or – rarely – actual metadata corruption. I've seen it on CentOS 7, Ubuntu 20.04, and RHEL 8. The origin LV goes inactive, and you're stuck wondering if your data's gone. It's not gone. Here's how to get it back.

Quick Fix (30 seconds)

Don't panic yet. Try the simplest thing first.

• Run lvchange -ay /dev/vg_name/lv_name. This activates the origin LV if it's inactive. If it works, you're done. If it says "Can't change snapshot origin" – move to the next step.
• Check if the merge is still pending: lvs -a -o name,attr,origin,merge_failed. If merge_failed shows nothing or empty, the merge never started. You'll need the moderate fix.

Skip this if you already tried activating the volume and got errors. Go straight to the moderate fix.

Moderate Fix (5 minutes)

The merge probably paused because the origin was mounted. Here's the real fix.

1. Find and unmount the origin: Run mount | grep vg_name to see if the origin LV is mounted. If it is, run umount /mountpoint. Wait 10 seconds.
2. Kill any processes using it: fuser -km /mountpoint. This closes file handles. I've seen a single open log file block the merge.
3. Try the merge again: lvconvert --merge vg_name/snapshot_name. You'll see something like "Merging of snapshot vg_name/snapshot_name started." Wait 30 seconds.
4. Activate the origin: lvchange -ay vg_name/origin_lv. If this works, run e2fsck -f /dev/vg_name/origin_lv to check the filesystem. Reboot if you can.

If the merge refuses to start (stays as "Merging" in lvs output for more than a minute), you might have a thin snapshot issue. Thin snapshots sometimes need a manual cancel: lvchange -an vg_name/snapshot_name then lvremove vg_name/snapshot_name. You'll lose the snapshot but keep the origin.

Real-world scenario: I had a dev server where a Docker container kept the origin mounted. After systemctl stop docker and repeating step 3, it worked instantly. Docker containers are sneaky like that.

Advanced Fix (15+ minutes)

You're here because the moderate fix didn't work – the origin LV is stuck inactive, and lvchange -ay throws "Can't change snapshot origin" or just hangs. Now we go deeper.

Step 1: Check for incomplete merge

Run dmsetup table and look for any snapshot-origin devices. If you see something like vg_name-origin_lv-real, the merge is partially done. Don't reboot – you'll lose data.

dmsetup table | grep snapshot

If there's a snapshot-origin device, the kernel's waiting. You can force-resume the merge:

1. dmsetup suspend vg_name-origin_lv
2. dmsetup clear vg_name-origin_lv
3. dmsetup resume vg_name-origin_lv

Then try lvchange -ay again. If that fails, you have metadata corruption.

Step 2: Repair LVM metadata

This is the nuclear option. Backup first if you can – vgcfgbackup vg_name saves the metadata to /etc/lvm/backup/.

vgcfgbackup vg_name
cp /etc/lvm/backup/vg_name /root/vg_name.backup

Now remove the snapshot manually:

1. lvchange -an vg_name/snapshot_name (force with -y if it complains)
2. lvremove -f vg_name/snapshot_name
3. lvchange -ay vg_name/origin_lv

If that still fails, you need to manually edit the backup file. Edit /etc/lvm/backup/vg_name and remove the entire snapshot_name section (look for snapshot in the volume entries). Then restore:

vgcfgrestore -f /etc/lvm/backup/vg_name vg_name

Reboot after that. The origin should be active. Run e2fsck before mounting.

Step 3: Last resort – rebuild the VG

If nothing works, you might have a corrupted VG metadata. This is rare – I've seen it once on a bad SSD. Use pvck to check each PV's metadata, then vgck to attempt repair. If you're at this point, you're better off restoring from a full backup of the PVs.

Honestly, if you're here, your best bet is to restore from your backup. LVM merges are fragile, and I've learned the hard way that snapshots are for short-term safety nets, not long-term backups.

Prevention for Next Time

Set this in /etc/lvm/lvm.conf: snapshot_autoextend_threshold = 100 and snapshot_autoextend_percent = 20. Also, never run lvconvert --merge while the origin is mounted. Script it to unmount first. Trust me.