Learn RAID - Performance Tuning

#Introduction

An array that "works" isn't necessarily an array that's "fast". Episode 18 covers performance tuning — how to choose the right parameters for your workload, and more importantly, how to measure the impact objectively.

Storage performance is full of myths and guesses. The only way to know the right settings is to measure before and after each change with a benchmarking tool like fio. This episode teaches the parameters you can change and how to measure them without guessing. Let's start with the chunk size.

#Chunk Size

#What the Chunk Size Does

Chunk size is the size of the data piece written to one disk before moving to the next disk on a parity array. It affects two things: the stripe size and the I/O pattern of each disk.

View the array's chunk size

sudo mdadm --detail /dev/md0 | grep -i chunk

The mdadm --detail output shows Chunk Size : 512K or another value. A large chunk makes sequential writes efficient; a small chunk spreads the load more evenly for random I/O.

#Adjusting the Chunk

The chunk can be changed with a reshape (discussed in episode 7):

Change the chunk size

sudo mdadm --grow /dev/md0 --chunk=64

The practical rules:

Large sequential (media server, archives): chunk 512K to 1M.
Random and small writes (database, VM): chunk 64K to 256K.
Stripe cache holds active stripes; make sure it's not too small.

Test several values with a benchmark before settling on a choice.

#Stripe Cache and Parity Algorithm

#Stripe Cache

The stripe cache is the kernel buffer for stripes being built on a parity array. Its size can be adjusted through sysfs:

Check and set the stripe cache

cat /sys/block/md0/md/stripe_cache_size
echo 8192 | sudo tee /sys/block/md0/md/stripe_cache_size

The cat /sys/block/md0/md/stripe_cache_size output shows the number of cache pages. Enlarging the cache can improve parity write throughput, but uses more RAM — set it according to the system's free memory.

#RAID 5 and 6 Parity Algorithms

The kernel has several algorithms for computing RAID 5/6 parity, including raid6_algos which picks the best implementation for the CPU:

View available RAID 6 algorithms

cat /sys/module/raid6/parameters/raid6_algos

The output lists algorithms like neon gen2 gen1 sse2 sse1.... The kernel automatically chooses the fastest one for your CPU. On parity-heavy systems this choice matters; in general the default is already good.

#I/O Scheduler

#Choosing a Scheduler

The I/O scheduler determines how read-write requests are ordered toward the disk. For RAID arrays, the common choices:

mq-deadline: balances latency and throughput — a safe choice for HDDs.
none: straight to the device without scheduling — best for SSDs and NVMe.
bfq: per-process priority — useful for interactive desktops.

Check and change the scheduler

cat /sys/block/md0/queue/scheduler
echo mq-deadline | sudo tee /sys/block/md0/queue/scheduler

The cat /sys/block/md0/queue/scheduler output shows the active scheduler. Change it via echo for a temporary setting; for a permanent one, configure it through a udev rule or sysfs.conf.

#Queue Depth and Read-ahead

Two other parameters that are often changed:

Set read-ahead and nr_requests

sudo blockdev --setra 65536 /dev/md0
echo 128 | sudo tee /sys/block/md0/queue/nr_requests

The blockdev --setra command sets read-ahead in sectors, and nr_requests limits the number of queued requests. Values too small limit sequential throughput; values too large increase latency.

#Benchmarking with fio and dd

#fio: A Comprehensive Benchmark

fio is the most reliable storage benchmarking tool. An example for measuring random write IOPS:

Benchmark random write with fio

fio --name=randwrite --ioengine=libaio --rw=randwrite --bs=4k \
    --size=1G --numjobs=4 --iodepth=32 --direct=1 --filename=/mnt/raid/tes

The fio command above runs 4K random writes with a depth of 32. The output shows IOPS and lat — these are the numbers to compare across configurations.

#dd: A Quick Measurement

For a quick sequential throughput measurement:

Measure write throughput with dd

dd if=/dev/zero of=/mnt/raid/tes bs=1M count=2048 oflag=direct status=progress

The dd command above writes 2GB of data with oflag=direct to bypass the page cache, then shows the speed at the end.

Warning

Benchmarking affects real performance: random writes drain SSD endurance and full writes stress the disks. Always use a dedicated benchmark area (not production data), run outside peak hours, and pause between tests so results don't contaminate each other.

#A Disciplined Tuning Flow

Follow this flow so tuning doesn't become guesswork:

Measure a baseline with fio on the initial configuration.
Change one parameter at a time.
Measure again with the exact same benchmark.
Record the results — only changes with significant impact are kept.
Test on the real workload before locking in the configuration.

#Conclusion

Episode 18 gave you control over array performance: chunk size, stripe cache, parity algorithm, I/O scheduler, and read-ahead — all measured with fio and dd. You can now build an array that's fast for your workload, not just one that works.

Key takeaways:

Large chunk for sequential, small chunk for random I/O.
Stripe cache and parity algorithm affect parity write throughput.
mq-deadline scheduler for HDDs, none for SSDs and NVMe.
fio is the benchmark standard for IOPS and latency.
Change one parameter and re-measure — don't change several at once.
Always measure a baseline before any tuning.

In the next episode, episode 19, we'll discuss advanced ZFS RAID (BRT, Tiering) — the Block Reference Table for fast dedup, stable RAIDZ expansion, mature dRAID, and tiering with special vdevs and hybrid flash plus HDD. You'll explore ZFS's latest capabilities for modern storage.