Learn Multigress - Performance Tuning & Scalability

#Introduction

A slow gateway makes every service behind it slow too. Episode 15 covers performance tuning and scalability of Multigress: tuning resources and concurrency, optimizing route evaluation and backend connections, and scaling the gateway horizontally with autoscaling.

This episode's goal is simple: the gateway should handle as much traffic as possible with as few resources as possible, without sacrificing latency.

#Resource Sizing and Concurrency

#Determining CPU and Memory

Start with reasonable values, then measure. Gateway resource configuration is done through Helm values and can be tuned without changing code.

helm upgrade multigress multigress/multigress \
  --namespace multigress-system \
  --set gateway.resources.requests.cpu=500m \
  --set gateway.resources.requests.memory=512Mi \
  --set gateway.resources.limits.cpu=2000m \
  --set gateway.resources.limits.memory=2Gi

A 500m CPU request guarantees a CPU share in the scheduler, while the 2000m limit caps maximum usage. Requests that are too small make pods share CPU with each other and cause unstable latency.

#Concurrency and Worker Threads

The Envoy-based data plane processes requests with worker threads. The ideal worker count is close to the number of node cores, since too few threads waste CPU and too many trigger scheduler contention.

helm upgrade multigress multigress/multigress \
  --namespace multigress-system \
  --set gateway.concurrency=4

This concurrency value is injected as the number of worker threads. After changing it, monitor P95 latency under normal load: if it drops when workers are added, the gateway previously had too few threads.

#Optimizing Route Evaluation and Backend Connections

#Connection Pool

Opening a new TCP connection for every request is expensive. Multigress uses a connection pool to reuse connections to the backend, and you can tune its limits.

apiVersion: gateway.multigress.io/v1
kind: BackendTrafficPolicy
metadata:
  name: backend-pool
  namespace: platform
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: api-route
  connectionPool:
    tcp:
      maxConnections: 512
      connectTimeout: 5s
    http:
      h2MaxConcurrentStreams: 100
      maxRequestsPerConnection: 1000

maxConnections: 512 caps the simultaneous TCP connections to one backend, while h2MaxConcurrentStreams controls how many HTTP/2 streams fit in a single connection. This configuration protects the backend from spikes while keeping connection reuse healthy.

#Timeouts to Protect the Backend

Timeouts prevent stuck requests from hanging onto connections. Combining a request timeout and an idle timeout keeps the pool healthy.

apiVersion: gateway.multigress.io/v1
kind: BackendTrafficPolicy
metadata:
  name: backend-timeouts
  namespace: platform
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: api-route
  timeouts:
    requestTimeout: 10s
    idleTimeout: 60s

The requestTimeout: 10s value cuts off requests that exceed 10 seconds, and idleTimeout closes connections idle for too long. A slow backend can never drain the pool forever.

#Horizontal Scaling and Autoscaling

#Manual Scaling and HPA

Start with manual scaling to test the configuration, then switch to a HorizontalPodAutoscaler so pod count follows load.

kubectl scale deployment multigress-gateway -n multigress-system --replicas=4
kubectl rollout status deployment multigress-gateway -n multigress-system

#CPU-Based Autoscaling

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: multigress-gateway
  namespace: multigress-system
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: multigress-gateway
  minReplicas: 2
  maxReplicas: 8
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

The HPA adds replicas when average CPU utilization passes 70 percent and shrinks them back as load drops. The minReplicas: 2 bound preserves minimum availability, and maxReplicas: 8 caps cost.

#Benchmarking to Find Bottlenecks

Tuning changes must be proven under load. A simple tool like hey is enough to tell better configurations apart.

hey -n 10000 -c 200 http://api.example.com/health
kubectl get hpa multigress-gateway -n multigress-system

The hey -n 10000 -c 200 command presses the health endpoint with 200 parallel connections. While it runs, watch the HPA add replicas and compare the P95 latency before and after tuning.

#Closing

Episode 15 made your gateway ready to withstand load: resources and concurrency tuned based on measurement, connection pools and timeouts keeping backend connections efficient, and an HPA making capacity follow demand.

The key takeaways:

• Resource requests guarantee a share; limits cap maximum usage.
• The ideal worker thread count is close to the number of node cores.
• Connection pools reduce the cost of opening new connections.
• Timeouts prevent stuck requests from hanging onto the pool.
• A CPU-based HPA adds replicas when utilization passes the target.
• Benchmarking is needed to prove every tuning decision.

In the next episode 16 we'll discuss extensions & plugin ecosystem — integration with Envoy filters and custom plugins, advanced filter chains for request transformation, and leveraging third-party tools for security and observability. Your now-fast gateway will be extended with new capabilities.