Learn Keycloak - High Availability & Clustering

#Introduction

In episode 26 you managed clients at scale. Episode 27 settles the last question before production: what if one server dies? High availability (HA) & clustering keeps the authentication service running even when a node goes down. This is the material that separates a lab setup from a deployment users actually depend on.

#The Clustering Concept

Keycloak is designed to run as a cluster: several Keycloak instances share one database, share caches with each other, and are served behind a single entrance. The basic concepts:

• Active-active clustering — all nodes serve requests simultaneously; there's no "idle" node.
• Distributed cache (Infinispan) — Keycloak's cache runs on embedded Infinispan and is synchronized across nodes via JGroups.
• Session replication — user sessions are shared between nodes, so the next request can be served by any node.
• Load distribution — requests are spread evenly by the load balancer.

Why isn't a single node enough? Because one instance is a single point of failure: whatever the cause — crash, maintenance, or network — all logins stop. A common topology comparison:

#Cluster Setup

Starting a Keycloak cluster means running identical instances that know each other. The required steps:

1. Multiple instances — run two or more Keycloak nodes, ideally in different availability zones.
2. Shared database — all nodes must point to the same database. Without this, user and session data is inconsistent.
3. Cache replication — nodes communicate via JGroups to synchronize the Infinispan cache.
4. Discovery mechanism — nodes must find each other; KC_CACHE_STACK determines the discovery method per environment.

#Discovery Mechanisms

Each environment has a different way of making nodes aware of each other. On AWS, use the ec2 stack, which uses S3-based discovery:

kc.sh start --cache-stack ec2

KC_CACHE_STACK=ec2 configures JGroups so nodes in the same region discover each other and form a cluster automatically. On Kubernetes, use the kubernetes stack, which leverages DNS-based discovery. In a typical datacenter network, the default multicast-based mode is usually sufficient. Choose the stack that matches where Keycloak runs — don't force an AWS method in another environment.

#Load Balancing

A load balancer sits in front of the cluster. Two important decisions:

• Sticky sessions vs session replication — with session replication active, the load balancer doesn't need to pin users to a specific node; any node can serve the session. Without replication, sticky sessions become mandatory and a node that dies will drop all the sessions pinned to it.
• Health checks — the load balancer must know which nodes are healthy via the /health/ready (ready to serve), /health/live (process alive), and /health/started (startup complete) endpoints. Unhealthy nodes must be removed from rotation.

An NGINX configuration example:

NGINX load balancing with health check

upstream keycloak {
    server kc-01.example.com:8080;
    server kc-02.example.com:8080;
}
 
server {
    listen 443 ssl;
    server_name sso.example.com;
 
    location / {
        proxy_pass http://keycloak;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

The proxy headers — X-Forwarded-For and X-Forwarded-Proto — must be forwarded correctly, because Keycloak uses them to build URLs and process IPs (remember episode 24). SSL termination at the load balancer is also common: Keycloak sees a plaintext connection from the proxy, so make sure --proxy-headers is set so HTTPS links stay correct.

#Database Considerations

The database is the most sensitive point in the cluster:

• Database clustering — run a high-availability database (replication, failover) because it's the real single point.
• Connection pooling — size the pool to the number of nodes: too small causes queueing, too large burdens the database.
• Read replicas — can lower the read load, but sessions and in-flight writes must still go to the primary; never serve session data from a lagging replica.
• Database performance — because all nodes share the database, latency and SQL tuning impact the whole cluster directly.

Remember from the upcoming episode 28: the bigger the load, the bigger the role of database pool and cache tuning. Clustering adds nodes, but the database stays single — it's the real bottleneck most often.

The database used by all nodes also needs attention on connection timeout and idle connections: a quiet node still holds connections from the pool. Set the connection lifetime and idle limits so newly joining nodes don't struggle to get a slot in the middle of a spike.

#Testing HA

A cluster that isn't tested isn't high availability. The mandatory routines:

• Failover testing — deliberately kill one node, make sure users currently logged in don't lose their sessions and requests keep being served.
• Load testing — ramp up the load gradually, make sure new nodes join without disruption and responses stay stable.
• Disaster recovery drills — simulate the total failure of one region; measure how long the service takes to recover and how much data is lost (RTO and RPO detailed in episode 29).

#Closing

Episode 27 took Keycloak to an enterprise architecture: active-active clustering with Infinispan as the distributed cache, a shared database, inter-node discovery via KC_CACHE_STACK, load balancing with health checks, database considerations, and routine failover testing.

Key takeaways:

• The database is the real single point — a Keycloak cluster won't save you if the database isn't HA.
• Session replication frees the load balancer — without it, sticky sessions become mandatory and full of risk.
• Health checks are the load balancer's eyes — make sure /health/ready is used for routing decisions.
• A cluster must be tested to fail — failover, load, and DR drills are part of operations.

In the next episode (episode 28), you'll optimize that cluster's performance: performance tuning & monitoring — from the JVM, cache, to Prometheus metrics and Grafana dashboards.