Learn GraphQL - Horizontal & Vertical Scaling Strategies

#Introduction

Your traffic is rising — and the question isn't "will the server break", but "when and how to handle it". Episode 34 discusses scaling GraphQL: adding machine power, adding machine count, and making sure every component from database to WebSocket scales along.

We'll cover vertical scaling, horizontal scaling, database scaling, layered caching, scaling subscriptions, and resilience patterns like circuit breaker and bulkhead.

#Vertical Scaling

#Resources and Node.js Tuning

Vertical scaling adds resources to a single machine — the easiest way, but it has limits. Reasonable steps before moving horizontal:

• Increase the instance's CPU and memory.
• Limit HTTP request size and GraphQL payload size.
• Set Node.js memory and concurrency limits.

{
  "scripts": {
    "start": "node --max-old-space-size=2048 dist/index.js"
  }
}

Other optimizations: DataLoader batching (episode 9), caching (episodes 19-20), and trimming complex queries (episode 15) reduce per-request load so a single machine can serve more.

#Horizontal Scaling

#Load Balancing and Stateless Design

Horizontal scaling adds more instances behind a load balancer:

client -> load balancer -> instance 1
                        -> instance 2
                        -> instance 3

The main requirement: the server must be stateless — don't store sessions or cache in an instance's local memory. All shared state goes to Redis:

• Query and field cache: Redis (episode 20).
• Authentication state: JWT (stateless by nature, episode 13).
• Subscriptions: Redis PubSub so events spread across instances.

Run multiple instances with the cluster pm2 start dist/index.js -i max.

A load balancer (nginx, ALB, or Kubernetes Service) distributes requests. Consistency is achieved because there's no local state to keep in sync.

#Database Scaling

#Read Replicas and Connection Pooling

The database is often the first bottleneck. Strategies:

• Read replicas: separate reads (GraphQL queries) from writes (mutations) by directing reads to a replica.
• Connection pooling: each instance opens many connections; a pooler like PgBouncer centralizes connections to the database.
• Sharding: splitting data across several databases by key — complex, so it's the last resort.

docker run -d --name pgbouncer \
  -e DB_HOST=db-primary -e DB_USER=app -e DB_PASSWORD=pass \
  -p 6432:5432 edoburu/pgbouncer

App connections point at PgBouncer (localhost:6432) instead of the database directly. This is especially important in serverless environments (episode 37) where connections are opened and closed constantly.

#Caching Layers and WebSocket Scaling

#Multi-Tier Caching

Cache at several layers to cut database load:

• CDN: cache public queries at the edge (episode 20).
• Redis cluster: distributed app cache, with eviction and TTL.
• Database cache: repeated access patterns are trimmed at the query level.

For invalidation at scale, use an event-based pattern: mutations publish an event (episode 16) and the cache is purged centrally, rather than relying on TTL alone.

#WebSocket and Subscription Scaling

WebSocket-based subscriptions need special attention:

• Sticky sessions so a WebSocket connection stays on the same instance, or
• Redis PubSub (episode 16) which spreads events to all instances — more robust because it doesn't rely on instance affinity.

Redis PubSub across instances

import { RedisPubSub } from "graphql-redis-subscriptions";
 
const pubsub = new RedisPubSub({
  publisher: new Redis(process.env.REDIS_URL),
  subscriber: new Redis(process.env.REDIS_URL),
});

WebSocket connections are managed with graphql-ws (episode 16) on each instance; events from any mutation spread through Redis so all clients receive them. Also limit connections per user and use heartbeats to clean up dead connections.

#Performance at Scale

#Circuit Breaker and Bulkhead

At large scale, one component's failure must not take everything down:

• Circuit breaker: if a dependency (for example a payment API) fails repeatedly, "open" the circuit and fail fast without waiting for timeouts.
• Bulkhead: separate connection pools per service, so a slow service doesn't exhaust all connections.

Bulkhead for the database

import { Pool } from "pg";
 
const userPool = new Pool({ connectionString: DB_URL, max: 10 });
const orderPool = new Pool({ connectionString: DB_URL, max: 5 });

These resilience patterns will be deepened in episode 35 together with high availability strategies.

#Conclusion

Key takeaways:

• Vertical scaling is fast but limited; horizontal scaling is the long-term path.
• Stateless design with Redis is a prerequisite for horizontal scaling.
• Read replicas and PgBouncer lighten database load.
• Layered caching cuts load at every layer.
• Redis PubSub distributes subscriptions across instances.
• Circuit breaker and bulkhead maintain resilience at scale.

In the next episode, episode 35, you'll learn about disaster recovery and high availability — multi-region design and failover, backup strategies with point-in-time recovery, disaster recovery planning with RTO and RPO, the circuit breaker pattern, and graceful degradation with feature flags. Your API will be resilient!