Learning Caddy - Load Balancing Strategies

#Introduction

A single application isn't always enough. When traffic grows, you add a second instance, a third, and so on — and then you need something that distributes requests fairly among them. That's load balancing, and Caddy has it built right into the reverse_proxy directive.

#Load Balancing in reverse_proxy

#Multiple Backends

Registering multiple backends is as easy as writing several addresses:

app.example.com {
    reverse_proxy localhost:8080 localhost:8081
}

Caddy distributes requests automatically. The default policy is round_robin — each request goes to the next backend in turn.

#Automatic Load Distribution and Failover

• Automatic: requests are divided according to the policy with no extra configuration.
• Health check: backends that fail checks are temporarily removed from the pool.
• Failover: if one backend doesn't respond, requests are routed to another backend.

#Load Balancing Policies

#round_robin and least_conn

A policy determines how a request picks a backend:

app.example.com {
    reverse_proxy {
        to localhost:8080 localhost:8081 localhost:8082
        lb_policy least_conn
    }
}

• round_robin (default): even alternation.
• least_conn: picks the backend with the fewest connections — ideal for long-running requests.

#Hash and Sticky Policies

Some policies target specific requests to the same backend:

• ip_hash: requests from the same IP go to the same backend — simple session affinity.
• uri_hash: hashing based on the URI path — consistent for caching.
• header: hashing based on a specific header.
• cookie: sticky sessions with a cookie — covered in episode 15.
• random and random_choose: random selection, for distributions that don't need to be fair.

ip_hash example:

app.example.com {
    reverse_proxy {
        to localhost:8080 localhost:8081
        lb_policy ip_hash
    }
}

lb_policy ip_hash ensures the same client always reaches the same backend — useful for applications with light state.

#Health Checks

#Active Health Checks

Caddy probes backends periodically:

app.example.com {
    reverse_proxy {
        to localhost:8080 localhost:8081
        health_uri /healthz
        health_interval 10s
        health_timeout 5s
    }
}

• health_uri /healthz — the path that gets probed.
• health_interval 10s — probe every 10 seconds.
• health_timeout 5s — a backend is considered failed if it doesn't respond within 5 seconds.

#Passive Health Checks

Without probing, Caddy judges health from actual request responses:

app.example.com {
    reverse_proxy {
        to localhost:8080 localhost:8081
        max_fails 3
        fail_duration 30s
    }
}

#Upstream Configuration

#Weights and Request Limits

Not all backends are equally strong. Give weights so bigger backends receive more:

app.example.com {
    reverse_proxy {
        to localhost:8080 localhost:8081 localhost:8082
        lb_policy round_robin
    }
}

Weights are written as a list with options:

app.example.com {
    reverse_proxy {
        to http://localhost:8080 http://localhost:8081
        dial_timeout 10s
        max_conns_per_host 500
    }
}

#Response Header Timeout

For slow backends:

app.example.com {
    reverse_proxy {
        to localhost:8080
        transport http {
            response_header_timeout 30s
        }
    }
}

#Use Case: Scaling and High Availability

#Horizontal Scaling

• Run several application instances behind a single Caddy domain.
• When traffic rises, add an instance — just add one line to the Caddyfile.
• When an instance goes down, the health check removes it automatically.

#Zero-Downtime Deployment

• Deploy a new version by adding a new instance before removing the old one.
• The health check ensures the new instance is ready before it receives traffic.
• caddy reload applies pool changes without dropping connections.

caddy reload --config Caddyfile reloads the configuration seamlessly — a pattern we'll use repeatedly in operations.

#Conclusion

Episode 14 opened up Caddy's load balancing strategies: registering multiple backends, policies like round_robin, least_conn, ip_hash, and random, active health checks with health_uri and passive ones with max_fails, and upstream configuration with weights, timeouts, and connection limits.

Key takeaways:

• reverse_proxy with multiple to addresses distributes requests automatically.
• lb_policy selects the distribution method, default round_robin.
• Active health checks use health_uri and health_interval.
• Passive health checks use max_fails and fail_duration.
• dial_timeout and max_conns_per_host manage the load per backend.
• Load balancing is the foundation of horizontal scaling and zero downtime.

In the next episode, episode 15, we'll cover session persistence & sticky sessions — cookie-based affinity, IP and header affinity, cookie configuration with secure and httponly attributes, when to use sticky sessions, and stateless alternatives like shared session storage and JWT.