Learn Linux Filesystem - Repair & Recovery

#Introduction

No filesystem is immune to damage. Power loss, worn-out disks, or kernel bugs can make a filesystem unmountable. Episode 11 covers what to do when the worst happens: the correct order of actions, each filesystem's repair tools, and when to stop trying on your own.

The first recovery principle to hold on to: don't make things worse. Every repair tool can overwrite data that could still be saved if you take a wrong step. That's why backing up the disk's state before any repair is always step zero.

We cover repair for ext4, XFS, btrfs, and ZFS, then move into physical data recovery with ddrescue. By the end of the episode, you'll know when to bring in professionals.

#Offline vs Online Repair

#Offline Repair

Offline repair requires the filesystem to be unmounted — the filesystem must not be used during the check because the tool modifies structures directly:

• ext4: e2fsck / fsck.ext4.
• XFS: xfs_repair.
• btrfs: btrfs check --repair.

The same rules apply: unmount first, and ideally boot into a live environment so no process touches the disk.

#Online Repair

Modern filesystems offer verification and repair without unmounting:

• btrfs: btrfs scrub — reads data, verifies checksums, repairs from healthy copies.
• ZFS: zpool scrub — the same function, plus self-healing from mirror/parity.

Online repair is safe because it runs on checksum-verified blocks — the tool knows which data is bad and can pick the correct source.

#Repair per Filesystem

#e2fsck for ext4

e2fsck checks and repairs ext4 structures. Run it while unmounted:

sudo umount /mnt/lab
sudo e2fsck -f -y /dev/loop0

-f forces a full check, -y answers yes to all questions. If the primary superblock is damaged, use a backup:

sudo e2fsck -b 32768 /dev/loop0

The backup superblock block number can be checked with mke2fs -n on an identical filesystem. For heavy damage, this -b is often a lifesaver.

#xfs_repair for XFS

xfs_repair must run while the filesystem is unmounted — forcing it while mounted will cause damage:

sudo xfs_repair -n /dev/loop0
sudo xfs_repair /dev/loop0

-n is a dry-run that only reports without changing anything. Always run a dry-run first to assess the level of damage before real action.

#btrfs check --repair

btrfs has btrfs check for offline checks and --repair to fix:

sudo umount /mnt/lab
sudo btrfs check /dev/loop0

btrfs check --repair is a last-resort and dangerous tool — it can introduce additional corruption on newer kernels. Before using it, always export every file that's still readable and prefer restoring from a snapshot.

#zpool scrub and zpool import

ZFS doesn't use fsck. Pool health is maintained by scrub, and recovery after a lost pool is done via import:

sudo zpool import -D labpool
sudo zpool scrub labpool
sudo zpool status labpool

zpool import -D shows detected pools without activating them. The CKSUM and REPAIR columns in the status show whether the scrub succeeded in recovering blocks.

#Physical Recovery: ddrescue

#Creating an Image of a Failing Disk

Before any tool touches a suspicious disk, create a full image of it with ddrescue. This tool reads good blocks first and retries bad blocks later:

sudo ddrescue /dev/sdb /media/save/disk.img /media/save/mapfile

The mapfile records which blocks were read successfully, so a second attempt only focuses on the failed blocks:

sudo ddrescue -r 3 /dev/sdb /media/save/disk.img /media/save/mapfile

Once the image is complete, do all repair operations on the image, not the original disk — this preserves the surviving disk and prevents additional damage.

#When to Use Professional Services

Some situations must be handled by professionals immediately:

• The disk isn't detected at all or makes strange noises (symptoms of head failure).
• e2fsck/xfs_repair keep failing on the same blocks.
• The data is high-value and was never backed up.

A rule of thumb: if ddrescue can't read a disk cleanly after two attempts, the more you keep powering it on, the greater the risk of permanent loss. Stop and hand it to a data recovery service with a cleanroom.

#Conclusion

Repair and recovery are skills whose value only becomes clear in a crisis. By understanding the offline vs online boundaries, the tools for each filesystem, and the importance of imaging a disk before acting, you turn panic into a calm, measured procedure.

Key takeaways:

• Back up or image the disk first before running any repair.
• ext4, XFS, and btrfs offline repair require unmounting; btrfs scrub and ZFS scrub are online.
• xfs_repair -n as a dry-run first; btrfs check --repair is a dangerous last resort.
• ZFS uses zpool import and scrub, not fsck.
• ddrescue images a failing disk with good-block priority.
• Hand over disks that can't be read cleanly to recovery professionals.

In the next episode, episode 12, we enter advanced ext4 and XFS — tuning tune2fs for reserved blocks and hashes, inline data, 64-bit metadata checksums, then reflink cp --reflink and dax persistent memory on XFS. You'll start optimizing filesystems at a level most admins rarely visit.