Learn Traefik - Rate Limiting & Circuit Breaker

#Introduction

When your application starts getting busy, the problem is no longer "how to route traffic" but "how to survive the surge". Episode 12 covers four middlewares that maintain stability under pressure: RateLimit, InFlightReq, CircuitBreaker, and Retry.

These four middlewares are the foundation of resilience. RateLimit prevents one client from flooding the backend, InFlightReq limits in-flight requests, CircuitBreaker cuts the flow when the backend starts failing consecutively, and Retry transparently covers temporary failures. Let us learn when to use which.

#RateLimit

#Limiting Requests per Second

RateLimit limits the number of requests based on source. Two numbers you must understand: average (average requests per second) and burst (the spike allowed before enforcement). The rateLimit middleware configuration in YAML:

http:
  middlewares:
    api-ratelimit:
      rateLimit:
        average: 20
        burst: 40
        period: "1s"
        sourceCriterion:
          ipStrategy:
            depth: 1

• average: 20 means an average of 20 requests per second per source.
• burst: 40 allows a burst of 40 requests in a short window before blocking.
• period: the calculation time window (default 1 second).
• sourceCriterion.ipStrategy: determines what counts as a "source". The default is the client IP; depth: 1 is useful if Traefik is behind another proxy.

RateLimit rejects requests that exceed the limit with status 429 Too Many Requests. This is the primary defense against abuse of public endpoints.

#InFlightReq

#Limiting In-Flight Requests

InFlightReq limits the number of requests being processed at the same time. Unlike time-based RateLimit, InFlightReq prevents simultaneous overload:

http:
  middlewares:
    limit-concurrent:
      inFlightReq:
        amount: 100
        sourceCriterion:
          requestHeaderName: "X-API-Key"

• amount: 100: only 100 requests may be processed in parallel from the same source.
• sourceCriterion.requestHeaderName: uses a specific header as the source identity — useful for APIs with API keys.

When the number of concurrent requests exceeds amount, requests are rejected with 429. Use InFlightReq to protect expensive resources such as worker queues or limited database connections.

#CircuitBreaker

#Cutting the Flow When the Backend Is Sick

CircuitBreaker follows the classic circuit breaker pattern: if the backend starts failing past a certain threshold, traffic flow is stopped so the backend can recover:

http:
  middlewares:
    api-breaker:
      circuitBreaker:
        expression: "NetworkErrorRatio() > 0.5 || ResponseCodeRatio(500, 600, 0, 600) > 0.3"

Expressions are built from the following functions:

• NetworkErrorRatio(): the ratio of network errors to total requests.
• ResponseCodeRatio(code1, code2, min, max): the ratio of responses within a status code range.
• LatencyAtQuantileMS(50): latency at a given quantile.

When the expression evaluates to true, the circuit opens: Traefik returns 503 for all requests during the checkPeriod. After the time window passes, the circuit enters half-open mode and tests again. The expression above opens if more than half of requests fail at the network level or more than 30 percent of responses have 5xx status — the two clearest signals that a backend is dying.

#Retry

#Automatic Retries

Retry tries a request again when the first backend server fails to respond:

http:
  middlewares:
    retry-lb:
      retry:
        attempts: 3
        initialInterval: "500ms"

• attempts: 3: total attempts including the first request.
• initialInterval: 500ms: the initial pause before the next attempt, increasing exponentially each time.

Retry covers temporary failures — container restarts, brief connection drops — so users feel nothing. But be careful with non-idempotent requests such as POST: retrying can double side effects. Do not put automatic retry on payment endpoints without an idempotency key.

#Closing

Key takeaways:

• RateLimit limits requests per second with average and burst.
• InFlightReq limits parallel in-flight requests from one source.
• CircuitBreaker opens the flow based on NetworkErrorRatio and ResponseCodeRatio.
• Retry retries requests with exponentially increasing pauses.
• Avoid retry on non-idempotent requests such as payments.
• All four work together: limit, cut, break, then try again.

In episode 13 next we will cover compression & buffering — the Compress middleware for gzip and brotli, Buffering for limiting request and response sizes, and ContentType for automatic content type detection. These middlewares touch performance and bandwidth efficiency on the wire.