Learn Samba - Clustering & Scale-out (CTDB)

#Introduction

Up to episode 17, you have one great Samba server. Episode 18 answers the question that arises when one server isn't enough — or isn't allowed to die: how do you make several Samba servers work as one entity? The answer is CTDB (Clustered Trivial Database), the component that turns a group of Linux nodes into an enterprise NAS with high availability and scale-out. This is the topic that separates an ordinary file server administrator from a storage architect.

#Concept: Why SMB Is Hard to Cluster

#The Locking Problem

File sharing is easy to cluster as long as clients only read. The problem arises with locking: SMB lets clients lock files (oplock, byte-range lock). If two Samba nodes manage locks independently, node A doesn't know node B is locking the same file — and two clients can "believe" they each hold exclusive rights. Result: data corruption.

#The CTDB Solution

CTDB unifies the lock databases of all nodes into one distributed database (using record-based locking technology). Each node consults CTDB before locking; lock decisions are cluster-wide. This is exactly the analogue of a cluster file system at the SMB level — which is why CTDB is called Clustered Samba.

#CTDB Architecture

#Cluster Components

        Public IP 192.168.10.50
                 |
   +-------------+-------------+
   |             |             |
 node1          node2         node3
 smbd+ctdb      smbd+ctdb     smbd+ctdb
   |             |             |
   +---- shared storage (CephFS/GPFS) ----+

• Each node runs smbd + ctdbd and mounts shared storage (a cluster filesystem: CephFS, GPFS, GFS2, or GlusterFS).
• The public IP (called IP takeover) is actively owned by one node; when that node dies, the IP is moved automatically to another node — clients don't need to change addresses.
• ctdbd maintains heartbeats between nodes and manages failover.

#Why Storage Must Be Shared

The key to CTDB scale-out: all nodes see the same filesystem. There's no "file on node 1" vs "file on node 2" — there's one namespace accessed by all nodes. CTDB doesn't copy data; it only unifies locks and failover. Without shared storage, you don't have a cluster — you have several separate servers with different IPs.

#Basic Configuration

#smb.conf on All Nodes

[global]
   workgroup = LAB
   security = user
   clustering = yes
   idmap config * : backend = tdb
   idmap config * : range = 3000-7999

clustering = yes tells Samba it's running under CTDB — locking and state are moved to the cluster. All nodes must use identical configuration for consistent behavior.

#ctdb.conf

public_addresses = /etc/ctdb/public_addresses

The per-node public IP list:

192.168.10.50/24 node1
192.168.10.51/24 node2

#Running and Checking the Cluster

sudo systemctl enable --now ctdb

Verify the cluster status:

sudo ctdb status
sudo ctdb ip
sudo ctdb nodes

• ctdb status: node list, roles (leader), and OK/DISCONNECTED status.
• ctdb ip: public IPs and the node actively holding each.
• ctdb nodes: cluster member list.

#Failover and Scale-out

#Testing Failover

Stop one node (or sudo ctdb disable node1) and watch the IP move:

sudo ctdb ip

When node1 goes down, its public IP is taken over by another node within seconds — clients using that IP keep running (active TCP connections may drop and must reconnect, but the service doesn't die). This is the essence of high availability: downtime measured in seconds, not hours.

#Scale-out

CTDB scales by adding nodes (up to the storage scalability limit) — each node adds connection capacity and CPU for serving clients, while data stays in one namespace on shared storage. For a multi-node enterprise NAS:

• Use CephFS or GPFS as shared storage — both scale out themselves, giving nearly unlimited capacity.
• Add CTDB nodes to increase SMB throughput without moving data.
• Monitor ctdb status and each node's load (episode 20).

#Common Pitfalls

• Node can't join: the firewall blocks CTDB port (4379) between nodes — open it in all directions.
• Inconsistent locks between nodes: clustering = yes isn't active on all nodes, or Samba versions differ.
• Public IP doesn't fail over: public_addresses is misformatted or heartbeats are disrupted — check ctdb status.
• Corrupted data after failover: the shared storage isn't truly clustered (e.g. plain NFS) — use a real cluster filesystem.

#Closing

Key takeaways:

• CTDB unifies SMB locks across nodes via a distributed database — the foundation of Samba clustering.
• Shared storage (CephFS/GPFS) is a prerequisite; CTDB doesn't copy data, it manages state.
• clustering = yes in smb.conf on all nodes; ctdb status/ctdb ip for monitoring.
• Failover via public IP takeover gives second-scale HA; scale-out by adding nodes.
• CTDB isn't for AD DC — that's the AD replication world, not clustering.

In episode 19 next, we'll cover Samba in Docker/Kubernetes — samba images like linuxserver and dperson, mounting volumes for state, K8s StatefulSet, and the correct pattern: stateless containers with exposed services. Modernizing the deployment without losing Samba's essence!