Learn RAID - RAID for SSD/NVMe

#Introduction

All of our discussion so far assumed HDDs. Episode 10 covers something many admins forget: RAID on top of SSDs and NVMe behaves differently. SSDs have flash cells with a limited number of writes, and RAID parity can accelerate flash aging if not configured correctly.

But solid-state media also opens up new possibilities: very low latency, TRIM for reclaiming space, and ZFS cache designs like SLOG and L2ARC. In this episode you'll learn to protect SSD lifespan while maximizing its performance inside an array. Let's start with TRIM.

#TRIM and Discard on Arrays

#What Is TRIM

When an HDD deletes a file, the old data can be overwritten at any time. SSDs can't simply overwrite — flash pages must be erased before being written again. TRIM tells the SSD which blocks are no longer in use so the controller can clean them up in the background. Without TRIM, SSDs slow down over time because they keep having to copy pages that still contain stale data.

#Discard in mdadm Arrays

mdadm forwards TRIM commands (called discard) to member disks as long as the bitmap feature is enabled:

sudo tune2fs -o discard /dev/md0
sudo fstrim -v /mnt/raid

The fstrim -v /mnt/raid command sends TRIM manually for all free space. Adding the discard option in tune2fs makes the kernel send TRIM automatically when files are deleted — for certain workloads, a periodic fstrim schedule is friendlier to performance.

#Trimming and Metadata

To make sure TRIM works across the array, check whether MD supports discard:

sudo mdadm --detail /dev/md0 | grep -i "discard\|bitmap"

The mdadm --detail output shows features like resync and a potential bitmap. mdadm enables discard automatically on metadata versions 1.1 and 1.2 — one of the reasons to choose modern metadata back in episode 4.

#Wear-leveling and Parity Overhead

#Endurance and Lifecycle

SSDs write data in large units internally (which affects endurance), even though the OS sends small writes. Wear-leveling is the controller's mechanism for spreading writes across all cells so no cell wears out quickly. The total amount of writes an SSD can handle is measured in TBW (Terabytes Written) — this value determines the SSD's lifespan.

#Parity Adds Writes

RAID 5 and 6 impose read-modify-write: every small write forces the system to read a block, compute parity, then write twice (data and parity). This doubles the number of physical writes an SSD must absorb. In ZFS, there's an additional cost: every block also stores checksums and ZFS metadata.

The result is two things to pay attention to:

• Parity arrays on SSDs age faster than HDDs under the same write load.
• RAID 10 and mirrors add fewer writes (only data duplication without parity calculation), so they're friendlier to flash lifespan.

#Longevity Tips

• Don't fill the SSD completely — leave over-provisioning space so the controller has room for wear-leveling.
• Use mirror or RAID 10 for write-heavy workloads.
• For parity on top of SSDs, make sure you understand the additional writes before deciding.

#NVMe Striping and Endurance

#NVMe Advantages

NVMe is a protocol designed for solid-state media, with hundreds of thousands of IOPS and microsecond latency. Combining several NVMe drives in an array can deliver gigabytes per second of throughput. However, on many systems NVMe devices have lower endurance than enterprise SATA SSDs — checking the TBW figure becomes mandatory.

#NVMe Striping

sudo mdadm --create /dev/md0 --level=0 --raid-devices=2 /dev/nvme0n1 /dev/nvme1n1

The mdadm --create --level=0 command above creates pure striping. RAID 0 provides no redundancy — suitable for scratch caches or data that can be rebuilt. For important data, consider a mirror or RAID 10 on top of NVMe even though it costs double the capacity.

#Cache and Log in ZFS: SLOG, ZIL, and L2ARC

#SLOG and ZIL

ZFS writes every synchronous write to the ZIL (ZFS Intent Log) to guarantee consistency, then batches it to the main disks in the background. By placing the ZIL on a fast device called SLOG (Separate Intent Log), you move the synchronous write load from the data disks to a faster device:

sudo zpool add tank log /dev/nvme1n1p1
zpool status tank

The zpool add tank log /dev/nvme1n1p1 command adds an NVMe partition as a log device. The zpool status output shows a separate logs section. A SLOG doesn't store permanent data — just a small mirror of synchronous writes, so it can be an NVMe drive with high endurance.

#L2ARC: Read Cache on SSD

L2ARC is an SSD-based read cache placed in front of HDD disks:

sudo zpool add tank cache /dev/nvme2n1
zpool status tank

The zpool add tank cache command adds an SSD as L2ARC. Data frequently read from HDDs is then served from the SSD with much lower latency. L2ARC is volatile across reboots — its contents don't survive; it only speeds things up after warming up.

#Battery-backed Write Cache vs NVDIMM

On hardware RAID, the write cache is protected by a battery or supercapacitor so cached writes aren't lost on power failure. In the ZFS world, the same role is played by a reliable SLOG — devices with power-loss protection are recommended, or NVDIMM for the lowest latency and crash-safe data.

#Conclusion

Episode 10 explained that RAID on top of SSDs and NVMe demands extra attention: enable TRIM, understand parity's impact on endurance, and leverage ZFS's cache design — SLOG for synchronous writes and L2ARC for reads — so your solid-state media stays both durable and fast.

Key takeaways:

• TRIM or discard is important for maintaining SSD performance inside arrays.
• RAID parity adds physical writes that accelerate flash aging.
• RAID 10 and mirrors are friendlier to endurance than RAID 5/6 for heavy writes.
• NVMe offers high IOPS, but you must check its TBW endurance figure.
• SLOG accelerates ZFS synchronous writes; L2ARC accelerates repeated reads.
• NVDIMM and power-loss protection keep the write cache safe during power failures.

In the next episode, episode 11, we'll discuss boot, partitioning & OS integration — placing /boot on an array, GRUB integration, GPT vs MBR, 4K alignment, and the luks-on-raid versus raid-on-luks ordering. You'll learn to build a system that boots directly from RAID.