Learning nginx - Load Balancing Algorithms & Upstream Management

#Introduction

One backend is never enough in production. When an application must serve thousands of requests per second or survive a server going down, you need load balancing. This Episode 7 covers how NGINX distributes traffic across multiple backends.

You'll define a backend cluster with the upstream block, pick the right load balancing algorithm for your needs, understand the differences between weighted round robin, least connections, IP hash, and generic hash, and leverage the max_fails, fail_timeout, backup, and down parameters.

#Defining a Backend Cluster

#The upstream Block

The upstream block groups several backend servers into one logical pool:

upstream backend_app {
    server 10.0.1.11:3000;
    server 10.0.1.12:3000;
    server 10.0.1.13:3000;
}
 
server {
    listen 80;
    server_name app.example.com;
 
    location / {
        proxy_pass http://backend_app;
    }
}

The name backend_app is used as the proxy_pass target. NGINX divides requests among the three backends according to the selected algorithm.

#Servers with Parameters

Each server entry in an upstream can use additional parameters:

upstream backend_app {
    server 10.0.1.11:3000 weight=3;
    server 10.0.1.12:3000 max_fails=2 fail_timeout=30s;
    server 10.0.1.13:3000 backup;
    server 10.0.1.14:3000 down;
}

• weight=3 — the backend receives three times the traffic share.
• max_fails and fail_timeout — passive health check settings.
• backup — only used if all primary servers are unavailable.
• down — the backend is marked offline without removing the configuration.

#NGINX Load Balancing Algorithms

#Round Robin (Default)

Without any configuration, NGINX uses round robin: requests are evenly distributed in turn to each backend. Simple and fair as long as all backends are homogeneous.

upstream backend_app {
    server 10.0.1.11:3000;
    server 10.0.1.12:3000;
}

#Weighted Round Robin

With weight, larger backends get a larger share:

upstream backend_app {
    server 10.0.1.11:3000 weight=3;
    server 10.0.1.12:3000 weight=1;
}

#Least Connections

The least_conn directive sends requests to the backend with the fewest active connections:

upstream backend_app {
    least_conn;
    server 10.0.1.11:3000;
    server 10.0.1.12:3000;
}

Suitable for requests with uneven durations, like file uploads or long polling.

#IP Hash

ip_hash binds a client to the same backend based on a hash of its IP. This provides session persistence without sticky configuration in the application:

upstream backend_app {
    ip_hash;
    server 10.0.1.11:3000;
    server 10.0.1.12:3000;
}

The same client is always directed to the same backend, as long as that backend is healthy.

#Generic Hash

hash uses a value we specify, such as the request URI, to choose the backend:

upstream backend_app {
    hash $request_uri consistent;
    server 10.0.1.11:3000;
    server 10.0.1.12:3000;
}

The consistent parameter keeps the mapping stable when the backend list changes — a ketama hashing technique that drastically reduces cache misses.

#Passive Health Checks and Server States

#Basic Concepts

NGINX open source performs passive health checks: a backend is considered failed after it fails to serve several requests within a given period:

upstream backend_app {
    server 10.0.1.11:3000 max_fails=3 fail_timeout=30s;
    server 10.0.1.12:3000 max_fails=3 fail_timeout=30s;
}

After 3 failures within 30 seconds, the backend is marked unhealthy and receives no traffic for the fail_timeout period. After that period, NGINX tries sending requests again — if successful, the backend automatically comes back online. This resilience is what made NGINX popular.

#Monitoring Load Balancing Results

Look at the error log to see upstream events:

sudo tail -f /var/log/nginx/error.log

tail -f shows backend up and down events. Combine it with repeated curl -I requests to test traffic distribution across all backends.

#Conclusion

Episode 7 completed load balancing: you can group backends in an upstream block, choose the appropriate algorithm, and apply passive health checks with max_fails, fail_timeout, backup, and down.

Key takeaways:

• The upstream block groups backends into one logical pool.
• Round robin is the default; weight is for backends with different capacities.
• least_conn is for requests with uneven durations.
• ip_hash provides session persistence; hash ... consistent stays stable as the cluster changes.
• Passive health checks use max_fails and fail_timeout.
• backup prepares a standby, down deactivates a backend.

In the next episode we'll discuss HTTP caching and microcaching — setting up proxy_cache_path, enabling caching with proxy_cache, reading cache status via X-Cache-Status, and microcaching strategies to survive sudden traffic spikes.