Learn NAS - High Availability & Scale-out

#Introduction

One reliable NAS is already great. Episode 20 takes you to the next level: high availability and scale-out — how to keep storage alive when one node dies, and how to add capacity beyond the limits of a single machine.

High availability (HA) answers the continuity question: if the primary server fails, a second system takes over without major downtime. Scale-out answers the capacity question: data that's too large or too heavily accessed for one node can be spread across many nodes.

By the end of this episode you'll be able to understand TrueNAS HA with failover and fencing, distinguish shared storage topologies, build object storage with MinIO and Garage, and recognize Ceph's role in distributed storage.

#TrueNAS HA

#The High Availability Concept

TrueNAS HA (available in the Enterprise line) provides two controllers sharing the same storage. If the primary controller fails, the second automatically takes over services. Clients barely notice the interruption.

midclt call failover.get_state

The midclt call failover.get_state command shows the failover state, for example MASTER, BACKUP, or FAULTED. This status determines which controller is serving.

#Shared Storage Topology

In an HA architecture, storage must be reachable by both controllers — usually via SAS or iSCSI toward a shared enclosure. This is what's called a shared storage topology: storage separated from compute, so both nodes access the same data without replication.

#Failover and Fencing

#Automatic Failover

Failover moves services from a failed node to a healthy one. TrueNAS HA monitors controller and service health; when a failure threshold is exceeded, the backup controller activates its interfaces and mounts storage.

#Fencing

Fencing is the mechanism that ensures a failed node is truly out of the system before another node takes over. Without fencing, two nodes could write to the same storage simultaneously and corrupt data. Fencing can be a hardware power-off or cutting off storage access.

It's important to understand: HA is not a backup. The same data can be corrupted in both copies because it comes from one source. Combining HA with offsite replication is still required for full protection.

#Scale-out with Clusters

#When Scale-out Is Needed

Scale-out is needed when one node isn't enough: maximum capacity reached, insufficient throughput, or higher availability required than one machine can provide. Scale-out adds nodes as a single distributed storage system.

#Common Scale-out Patterns

• Cluster filesystems like GlusterFS and CephFS.
• Object storage like MinIO and Garage.
• Active-active replication with two-way synchronization.

Each pattern has trade-offs between consistency, latency, and complexity. For a homelab, object storage is the most reasonable entry point.

#Object Storage: MinIO and Garage

#MinIO for S3-Compatible Storage

MinIO is an open-source object storage compatible with the Amazon S3 API. With MinIO, the NAS can serve buckets used by applications and backup tools. Data is stored as objects with metadata, not as a traditional filesystem.

docker run -d --name minio \
  -p 9000:9000 -p 9001:9001 \
  -v tank-minio:/data \
  minio/minio server /data --console-address ":9001"

The docker run command runs MinIO with data in the tank-minio volume. The management console is on port 9001, and the S3 API on port 9000.

#Garage as a Light Alternative

Garage is a distributed object storage designed for self-hosting and edge use. Lighter than MinIO at small scale, Garage can easily be combined into a multi-node cluster for cross-machine redundancy.

garage layout assign --force \
  --layout-version 1 \
  1a2b3c4d5e

The garage layout assign command assigns a node to the cluster layout. Once the layout is applied, buckets can be created and used through the S3 API.

#Ceph Integration

#Distributed Storage with Ceph

Ceph is an open-source distributed storage providing block, file, and object from a single cluster. Ceph is known as the engine behind many clouds and Kubernetes storage. However, its complexity is far beyond typical homelab needs.

ceph -s

The ceph -s output shows cluster health: HEALTH_OK status, usage, and OSD count. Ceph needs at least several nodes and a dedicated network to be stable.

#When to Use Ceph

Ceph is worthwhile if you manage a large-scale infrastructure that's already complex — for example, a Kubernetes cluster with many nodes needing distributed storage. For a homelab with one or two NAS units, ZFS replication and MinIO are far simpler and sufficient.

#Closing

In this episode 20 you understood availability and scale: TrueNAS HA with failover and fencing, shared storage topology, scale-out clusters, the MinIO and Garage object stores, and Ceph integration for distributed storage.

Key takeaways:

• HA automatically moves services to a second controller, but isn't a backup replacement.
• Fencing prevents two nodes from writing to the same storage simultaneously.
• The MinIO and Garage object stores are the easiest scale-out entry point.
• Ceph is powerful but complex; only for large-scale infrastructure.
• Choose complexity only when the need truly demands it.

In the next episode, episode 21, we'll cover modern features and roadmap — from TrueNAS SCALE 25.10 Goldeye and the 25.10.2 stable release, TrueNAS 26 with dataset tiering and OpenZFS 2.4, to OpenMediaVault 8.3 based on Debian 13 along with the direction of the ZFS and object storage ecosystem. Your system is mature; it's time to look ahead.