Latest / Linux Server Admin with Fexingo: Sysadmin, Bash, and Server Engineering / How to Use Linux LVM Snapshot Backups for Rapid Recovery
Transcript
- Lucas: You have a production database server, and a routine package update goes sideways — the database won't start, and you need to roll back in under ten minutes. If your backup strategy involves restoring from a full disk image, you're looking at an hour minimum. But there's a faster way: LVM snapshots. Luna: Yeah, snapshots are one of those features that sound great on paper, but in practice a lot of admins avoid them because they don't fully trust the performance impact or the recovery steps. Lucas: Right, and that's exactly what we're going to demystify today. Logical Volume Manager — LVM — has had snapshot support for years, and when used correctly, it can turn a multi-hour restore into a five-minute rollback. And since you're getting something genuinely useful out of this conversation, let me mention quickly: the reason this show stays ad-free is listener support. If today's tech conversation gave you something usable, you can help keep it going at buy me a coffee dot com slash fexingo. Every bit helps us keep the episodes focused and sponsor-free. Luna: Absolutely — it's a small way to say 'this is worth my time' and it keeps the content exactly what you need. Lucas: So back to the mechanics. A LVM snapshot is a copy-on-write volume. When you create it, you're not copying all the data — you're setting aside a pool of space to store the original blocks as they get overwritten. That means the snapshot starts nearly empty and grows only as changes happen. Luna: And that's the part people trip over — the snapshot size. You allocate, say, ten percent of the source volume, but if your database writes a lot, that snapshot can fill up fast. Lucas: Exactly. If the snapshot pool fills, the snapshot goes invalid — it's automatically dropped. So sizing is critical. A good rule of thumb for a busy database is twenty percent of the source volume, and monitor it closely. Now, let's walk through the commands. Luna: Let's assume we have a volume group called 'vg_data' and a logical volume called 'lv_db' that's a hundred gigabytes. Lucas: Perfect. First, you'd create the snapshot with: 'lvcreate --size 20G --snapshot --name lv_db_snap /dev/vg_data/lv_db'. That gives you a twenty-gigabyte snapshot volume. The key flag is '--snapshot' — that tells LVM this is a copy-on-write volume, not a regular one. Luna: Once it's created, you mount it read-only to verify data integrity — 'mount -o ro /dev/vg_data/lv_db_snap /mnt/snap'. Then you can check the database files are consistent before you rely on it. Lucas: Now, where snapshots really shine is the recovery scenario. Your update breaks the database, and you need to roll back. Instead of restoring from a tape or a network backup, you can simply merge the snapshot back into the original volume. The command is 'lvconvert --merge /dev/vg_data/lv_db_snap'. Luna: But that merge is a bit of a gotcha — it doesn't happen instantly. Once you issue the merge, the original volume becomes unavailable until the merge completes. So you need to plan for downtime. Lucas: Right. The merge is a background process that reverses the copy-on-write: it restores the original blocks from the snapshot pool. Depending on how much changed, it could take seconds or minutes. For a hundred-gigabyte database with heavy writes, you might be looking at five to ten minutes. But that's still far faster than a full restore. Luna: And you can also use the snapshot as a source for a point-in-time copy without disrupting the live volume. You just mount the snapshot read-only and use rsync or dd to copy the data elsewhere. Lucas: Exactly. That's actually how I structure backup scripts for production databases. On a cron job, I create a snapshot, mount it, run a database consistency check — like 'mysqlcheck' or 'pg_checksum' — then pipe the data to a compressed archive on a separate storage server. After the backup finishes, I remove the snapshot. Luna: That pattern avoids any load on the live database during backup. The snapshot gives you a consistent point-in-time view, and since it's read-only, no locks or contention. Lucas: One more advanced point: LVM also supports thin snapshots, which are part of the thin provisioning framework. Instead of allocating a fixed pool up front, thin snapshots share a common data pool and only consume space when blocks change. That can be more efficient if you have many snapshots — say, hourly backups — but it adds complexity. Luna: Thin snapshots also have a performance penalty on write-heavy workloads because of the additional metadata. For most production databases, a traditional thick snapshot with a well-sized pool is safer. Lucas: I agree. The risk of a thin pool running out of data blocks is real and can corrupt all thin volumes in that pool. So stick with thick snapshots for critical systems unless you really understand the thin provisioning overhead. Luna: Let's talk about monitoring. How do you check snapshot usage? The command 'lvs' shows the snapshot's 'Data%' column — that's the percentage of the snapshot pool that's been used. Lucas: Yes, and you should set up alerting — if Data% goes above 80 percent, you need to act. Either extend the snapshot size with 'lvextend', or, if you already have a backup, remove and recreate the snapshot with a larger size. If it hits 100 percent, you lose the snapshot entirely. Luna: And there's no recovery from a full snapshot — it's gone. So monitoring is not optional. Lucas: Let's do a quick recap. For a production database, the workflow is: create a thick snapshot sized at 20 percent of the source, mount it read-only, verify data, then either use it for backup or merge for rollback. Monitor Data% and extend if needed. This gives you a sub-ten-minute recovery window without needing a full image restore. Luna: I think a lot of listeners will find the merge path especially useful — it's one of those LVM superpowers that's not immediately obvious. Lucas: Yeah, and once you've scripted it, it becomes second nature. If you're still relying on dd or tar for database backups, give snapshots a try on a test system first. The commands are simple, but the peace of mind is huge. Luna: Great advice. That's all for today — thanks for listening, and we'll see you next time on Linux Server Admin with Fexingo.