Learning Redis - Redis Cluster (Horizontal Scalability & Sharding)

#Introduction

Sentinel in episode 12 keeps availability, but there's another limit: capacity. A single master can only store data that fits in one memory and handle writes limited to one process. Episode 13 covers Redis Cluster — Redis's answer to horizontal scalability.

Redis Cluster divides data across many nodes via sharding, still provides redundancy through replicas, and handles failover itself without a separate Sentinel. This is the architecture used when the dataset exceeds a single node or write throughput exceeds a single master's capacity. Let's dissect how it works.

#When Do You Need Redis Cluster

#Two Main Signals

A cluster is not the first decision — there are real operational costs and complexity. You need a cluster when one of these conditions occurs:

• The dataset doesn't fit on a single node: data exceeds one master's RAM capacity, or maxmemory is forced very high.
• Write throughput exceeds a single master: one single-threaded process no longer suffices for the write load.

If your needs are only read scaling and availability, Sentinel + replicas (episode 12) is simpler. A cluster is worth choosing when data or writes are the bottleneck.

#Redis Cluster Architecture

#16,384 Hash Slots

The cluster divides the keyspace into 16,384 hash slots. Each key is mapped to a slot based on its hash:

slot = CRC16(key) mod 16384
 
cluster node 1: slot 0 - 5460
cluster node 2: slot 5461 - 10922
cluster node 3: slot 10923 - 16383

Each master has one or more replicas for failover. If a master dies, the cluster automatically promotes its replica — without Sentinel, because the cluster has its own internal consensus.

#Clients and the Cluster

When a client calls a key on the wrong node, the node replies with MOVED containing the slot and the address of the correct node. Cluster-aware clients follow this redirect transparently. Modern libraries (ioredis, redis-py, go-redis) support cluster mode natively.

#6-Node Redis Cluster Setup

#Configuring Each Node

Every node needs cluster-enabled yes:

port 7000
cluster-enabled yes
cluster-config-file nodes-7000.conf
cluster-node-timeout 5000
appendonly yes

cluster-enabled yes turns on cluster mode. Repeat this configuration for ports 7000-7005 (three masters and three replicas).

#Creating the Cluster

With redis-cli --cluster create, a single command assembles everything:

redis-cli --cluster create \
  127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 \
  127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 \
  --cluster-replicas 1

redis-cli --cluster create sets up the first three nodes (7000-7002) as masters and the next three (7003-7005) as replicas — the --cluster-replicas 1 flag means one replica per master. Slots are divided automatically and evenly.

#Verifying the Cluster

redis-cli -p 7000 CLUSTER INFO
redis-cli -p 7000 CLUSTER NODES

CLUSTER INFO shows the cluster_state:ok status and slot distribution. CLUSTER NODES lists all nodes with their role, address, and held slots. These are the first two diagnostic commands you should memorize for clusters.

#Hash Tags

#Grouping Related Keys Together

In a cluster, multi-key operations (e.g. MSET a b, SINTER, or transactions across two keys) are only valid if all keys are in the same slot. Hash tags guarantee this: only the key part inside curly braces is used to compute the slot.

slot(CRC16("{user:1}:profile")) == slot(CRC16("{user:1}:orders"))

The keys {user:1}:profile and {user:1}:orders map to the same slot because CRC16 is computed only over user:1. With that, operations involving both keys can run:

redis-cli MSET "{user:1}:profile" "Arman" "{user:1}:orders" "3"
redis-cli GET "{user:1}:profile"

MSET "{user:1}:profile" "Arman" "{user:1}:orders" "3" writes two keys to the same slot — legal in a cluster because they're in one slot. Without hash tags, this command errors with CROSSSLOT Keys in request don't hash to the same slot.

#Cluster Operations

A few things to keep in mind:

• Resharding: move slots between nodes with redis-cli --cluster reshard — possible without downtime.
• Rebalancing: redis-cli --cluster rebalance levels out slots when new nodes join.
• Multi-key constraint: cross-slot commands fail with CROSSSLOT — design keys with hash tags from the start.
• Failover: when a master dies, its replica is promoted automatically; be aware of cluster-require-full-coverage (default: the cluster refuses queries if any slot has no owner).

#Summary

Episode 13 equipped you with Redis Cluster for horizontal scalability: dividing 16,384 hash slots across master nodes, a 6-node architecture with 3 masters + 3 replicas, when to move up to a cluster, and hash tags for multi-key operations.

Key takeaways:

• Move to a cluster when the dataset or write throughput exceeds a single master.
• The keyspace is divided into 16,384 hash slots via CRC16(key) mod 16384.
• redis-cli --cluster create ... --cluster-replicas 1 assembles a 6-node cluster.
• CLUSTER INFO and CLUSTER NODES are the first diagnostics.
• The hash tag {user:1}:... groups related keys into one slot.
• Cross-slot operations fail with CROSSSLOT — design keys from the start.
• Resharding can be done online; the cluster chooses its own replicas without Sentinel.

In the next episode, episode 14, we move on to Security: Authentication, ACL & Encryption — securing Redis from unauthorized access. You'll learn requirepass, granular user Access Control Lists, TLS in transit, and hardening with protected-mode and rename-command. Ready to lock down your server?