Learn WebSocket - Load Balancing WebSocket Connections

#Introduction

A single Node.js server can only serve tens of thousands of connections. As the application grows, you add servers — but WebSocket connections are different from ordinary HTTP requests. A WebSocket connection lives a long time, so once it is thrown to a server, it must stay on that server forever.

Episode 15 covers load balancing for WebSocket: why sticky sessions are needed, the difference between Layer 4 and Layer 7 load balancers, NGINX and HAProxy configuration, and how to keep servers healthy during deployment.

#The Load Balancing Challenge

#Why Long-Lived Connections Are Hard

An HTTP request finishes in milliseconds, so it can be thrown at any server. A WebSocket connection lasts for hours. If the load balancer sends the upgrade request to server A but the next frames to server B, the connection breaks.

curl -i -H "Connection: Upgrade" -H "Upgrade: websocket" \
  -H "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==" \
  -H "Sec-WebSocket-Version: 13" \
  http://localhost:8080

The command above sends an upgrade handshake. The load balancer must recognize this request and direct the entire connection to a single backend from handshake to close.

#Sticky Sessions

The key is sticky session or affinity: all frames from one connection go to the same server.

• Cookie-based: the load balancer attaches a cookie pointing to a specific server.
• IP-based: the client's IP address is hashed to a single server.
• Connection-based: the same TCP connection is always sent to the same server.

Without this, a stateful application like chat breaks: messages are scattered across many servers.

#Layer 4 vs Layer 7

#The Fundamental Difference

Load balancers work at two different layers.

Layer 4 (TCP)  : melihat alamat IP dan port, memforward byte mentah
Layer 7 (HTTP) : melihat isi request, bisa baca header Upgrade

Layer 4 is very fast and does not care about the protocol — it just forwards bytes. Unfortunately it cannot read the Upgrade header, so it cannot guarantee per-connection sticky sessions.

Layer 7 reads HTTP headers and understands the WebSocket handshake. It can set cookie- or header-based affinity, but it costs a bit more because it must parse the request.

#The Trade-Off

• Choose Layer 4 if you need maximum raw throughput and all backends are equivalent.
• Choose Layer 7 if you need routing control, for example intelligently balancing WebSocket connection loads.

#NGINX Configuration

#NGINX as a Reverse Proxy

NGINX is the most common choice for WebSocket in production because its configuration is simple.

location /ws {
    proxy_pass http://ws_backend;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header Host $host;
    proxy_read_timeout 3600s;
    proxy_send_timeout 3600s;
}

The two most important headers: proxy_set_header Upgrade $http_upgrade and proxy_set_header Connection "upgrade". Without them, the WebSocket connection fails immediately because NGINX does not forward the upgrade request. The 3600-second timeout prevents NGINX from cutting healthy idle connections.

#Upstream with Sticky

For multiple backends, define an upstream with the sticky option.

upstream ws_backend {
    ip_hash;
    server 10.0.1.1:8080;
    server 10.0.1.2:8080;
}
 
server {
    listen 80;
    location /ws {
        proxy_pass http://ws_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

ip_hash consistently routes clients with the same IP to the same backend. For a more even distribution, use the cookie-based sticky cookie option.

#HAProxy Configuration

#HAProxy Built for Long-Lived Connections

HAProxy is known to be very efficient for persistent connections like WebSocket.

frontend ws_front
    bind *:80
    mode http
    use_backend ws_back if { hdr(Upgrade) -i websocket }
 
backend ws_back
    mode http
    server node1 10.0.1.1:8080 weight 1
    server node2 10.0.1.2:8080 weight 1

The condition { hdr(Upgrade) -i websocket } routes requests carrying the Upgrade header to the dedicated WebSocket backend. HAProxy handles WebSocket connections natively without extra configuration, as long as the timeout client and timeout server values are set long (for example 3600 seconds) so connections living longer than an hour are not cut.

#Health Checks and Graceful Shutdown

#Connection Health Checks

The load balancer needs to know which servers are healthy. For WebSocket, the health check should be in the application, not just an open port.

WebSocket health check endpoint

const http = require("http");
 
const server = http.createServer((req, res) => {
  if (req.url === "/healthz") {
    res.writeHead(200, { "Content-Type": "application/json" });
    res.end(JSON.stringify({
      status: "ok",
      koneksi: wss.clients.size,
    }));
    return;
  }
  res.writeHead(404);
  res.end();
});

The /healthz endpoint reports the server status and the number of active connections. The load balancer marks a backend healthy only if this endpoint responds with 200.

#Zero-Downtime Deployment

When deploying, the old server must release connections gently, not be killed outright.

Graceful shutdown

process.on("SIGTERM", () => {
  wss.clients.forEach((ws) => {
    ws.close(1001, "server akan di-deploy");
  });
 
  server.close(() => {
    console.log("semua koneksi ditutup, server berhenti");
    process.exit(0);
  });
 
  setTimeout(() => process.exit(1), 10000).unref();
});

Code 1001 means going away. A healthy load balancer detects the outgoing connections and routes clients to the remaining backends, while clients with the reconnection logic from episode 12 move without the user noticing.

#Closing

Episode 15 explained why WebSocket load balancing is more complex than ordinary HTTP: long-lived connections demand sticky sessions, understanding of the Upgrade header, and special treatment in the load balancer.

Key takeaways:

• A WebSocket connection must stay on one backend from start to finish.
• Sticky sessions are achieved with cookies, IP hash, or connection affinity.
• Layer 7 can read the Upgrade header; Layer 4 only forwards bytes.
• NGINX needs the Upgrade and Connection headers to forward WebSocket.
• Long timeouts prevent the load balancer from cutting healthy connections; health checks and graceful shutdown enable zero-downtime deploys.

In the next episode we cover horizontal scaling with the Redis adapter: cross-server pub-sub, room synchronization, and a chat architecture serving thousands of servers at once.