Learn OpenBSD - relayd & Load Balancing

#Introduction

In episode 15 you broke down pledge and unveil — security from inside the process. Now we return to networking to build something production desperately needs: relayd, OpenBSD's built-in relay and load balancer.

In episode 9, httpd could be a simple reverse proxy. But for larger scale — multiple backends, health checks, centralized TLS termination — OpenBSD provides relayd. It's the same tool many operators use to build the front door of their services: powerful, lightweight, and battle-tested.

#What relayd Does

relayd works at two levels:

• Layer 4 (TCP): forwards TCP connections based on address and port — it doesn't care about the application protocol.
• Layer 7 (HTTP): understands HTTP — inspecting methods, paths, and capable of TLS termination.

The combination of both makes relayd suitable as a reverse proxy, load balancer, and TLS terminator all at once.

#Enabling relayd

As usual, the service is managed with rcctl:

rcctl enable relayd
rcctl start relayd
rcctl status relayd

The configuration is at /etc/relayd.conf. Before starting, always validate:

relayd -n

#Anatomy of /etc/relayd.conf

The relayd configuration is built from three parts: table (backends), redirect/relay (rules), and protocol (optional, for layer 7). A basic TCP load balancer example:

table <web> { 192.168.10.10, 192.168.10.11 }
relay www {
    listen on 10.0.0.10 port 80
    forward to <web> roundrobin check icmp
}

Requests to 10.0.0.10:80 are forwarded round-robin to two backends. check icmp is a simple health check: backends that don't reply are removed from rotation.

#Health Checks: The Heart of a Load Balancer

Health checks determine whether a backend is worthy of receiving traffic. Several types:

table <web> { 192.168.10.10, 192.168.10.11 }
protocol http_health {
    http protocol http
    match request path "/healthz"
    expect "ok"
}
relay www {
    listen on 10.0.0.10 port 80
    forward to <web> check http path "/healthz"
}

With check http path, relayd periodically sends an HTTP request to /healthz; backends that don't respond with a success code are considered dead and removed from the pool. This is far more accurate than a simple ping.

#TLS Termination at relayd

TLS termination puts the encryption at relayd and forwards plaintext traffic to backends. Using the certificates from episode 10:

table <web> { 192.168.10.10, 192.168.10.11 }
protocol https {
    tls keypair example.com
    tls { certificate "/etc/ssl/example.com.crt" key "/etc/ssl/private/example.com.key" }
}
relay https {
    listen on 10.0.0.10 port 443 tls
    protocol https
    forward to <web> check http path "/healthz"
}

Backends only speak HTTP on the internal network; encryption and certificates are handled entirely by relayd. This simplifies operations: you manage certificates in one place.

#Layer 7: Reverse Proxy with Headers

To modify requests before forwarding — for example adding the X-Forwarded-For header or rewriting a path — use the HTTP protocol:

protocol http_proxy {
    http protocol http
    match request header set "Host" value "app.example.com"
    match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
}
relay app {
    listen on 10.0.0.10 port 8080
    protocol http_proxy
    forward to <app> check http path "/"
}

This is the same pattern as the httpd reverse proxy in episode 9, but with the power of health checks and multi-backend load balancing.

#Complete Scenario: A Reliable Web Frontend

Combine all the capabilities in one production scenario:

table <web> { 192.168.10.10, 192.168.10.11, 192.168.10.12 }
protocol https_front {
    tls keypair example.com
    match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
}
relay www {
    listen on 10.0.0.10 port 443 tls
    protocol https_front
    forward to <web> roundrobin check http path "/healthz" retry 3
}

• Three web backends, chosen round-robin.
• HTTP health checks ensure only healthy ones are served.
• Centralized TLS termination.
• The X-Forwarded-For header tells the backend the real client address.

If one backend breaks, the health check removes it automatically; when it recovers, it rejoins. The service never stops.

#Monitoring relayd

Verify that the relay works and the backends are healthy:

rcctl status relayd
tail -f /var/log/daemon
relayctl show host

relayctl show host displays the status of each backend in the pool — the key to confirming the health checks are working. Logs are in /var/log/daemon.

#Closing

In episode 16 you built relayd: understanding the layer 4 and 7 architecture, arranging /etc/relayd.conf, implementing HTTP health checks, doing TLS termination, and assembling reverse proxy plus load balancing for a reliable web frontend.

Key takeaways:

• table defines the backends, relay forwards traffic, protocol handles layer 7.
• Health checks (icmp, tcp, http) keep the pool containing only healthy backends.
• TLS termination at relayd simplifies certificate management.
• relayctl show host and the daemon log are your main monitoring tools.

In the next episode, episode 17, we'll run OpenSMTPD & mail services — OpenBSD's native mail server, arranging /etc/mail/smtpd.conf, choosing maildir or mbox, configuring relay, virtual users, STARTTLS, and DKIM and SPF integration.