Learn Envoy Proxy - High Availability & Scaling

#Introduction

After mastering the features inside one Envoy, it's time to think about many Envoys. Episode 17 covers high availability and scaling: sidecar, gateway, and standalone deployment patterns, keeping the xDS control plane available, and considerations when Envoy spreads across many zones and clusters. The key concept: Envoy is stateless, so scaling it is just about adding instances and keeping config consistent.

#Deployment Patterns: Sidecar, Gateway, and Standalone

#Sidecar per Workload

The most common pattern in a service mesh: one Envoy attached to every pod or VM workload.

apiVersion: v1
kind: Pod
metadata:
  name: orders-5f9d6b
spec:
  containers:
    - name: orders
      image: registry.example.com/orders:1.4.0
      ports:
        - containerPort: 8080
    - name: envoy
      image: envoyproxy/envoy:v1.31.0
      ports:
        - containerPort: 15006
      volumeMounts:
        - name: envoy-config
          mountPath: /etc/envoy
  volumes:
    - name: envoy-config
      configMap:
        name: orders-envoy-config

The envoy sidecar pattern makes every inbound and outbound orders request pass through Envoy. Benefits: isolation and per-workload policy. Cost: per-pod resource overhead.

#Gateway at the Edge

Unlike the distributed sidecar, a gateway centralizes Envoy at the entry point:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: edge-gateway
spec:
  replicas: 3
  selector:
    matchLabels:
      app: edge-gateway
  template:
    metadata:
      labels:
        app: edge-gateway
    spec:
      containers:
        - name: envoy
          image: envoyproxy/envoy:v1.31.0
          ports:
            - containerPort: 10000
            - containerPort: 9901

The edge-gateway deployment with replicas: 3 gives you three gateway instances behind a LoadBalancer. Because Envoy is stateless, adding a replica just adds one pod — there's no state replication to maintain.

#Standalone for Special Tasks

The standalone pattern uses Envoy for specific jobs: database proxies, egress proxies, or telemetry aggregators. It's not tied to a workload and doesn't have to be at the edge — Envoy stands alone with a dedicated config.

#High Availability of the xDS Control Plane

#More Than One Control Plane

If all Envoys depend on a single xDS control plane, that control plane is a single point of failure. The solution: run several instances behind a load balancer:

static_resources:
  clusters:
    - name: xds_cluster
      connect_timeout: 1s
      type: STATIC
      lb_policy: ROUND_ROBIN
      http2_protocol_options: {}
      load_assignment:
        cluster_name: xds_cluster
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: cp-1.internal
                      port_value: 18000
              - endpoint:
                  address:
                    socket_address:
                      address: cp-2.internal
                      port_value: 18000

With lb_policy: ROUND_ROBIN, Envoy alternates connections between cp-1 and cp-2. If one control plane dies, Envoy opens a new stream to the available instance — config already received stays in use.

#Behavior When the Control Plane Is Down

A point that's often misunderstood: an Envoy that loses its xDS connection does not stop. It keeps running the last config it received; only the ability to receive changes is lost. This is why Envoy is designed stateless — traffic keeps flowing, only updates are delayed.

#Safe Drain and Restart

When an Envoy is being replaced, the drain process is critical:

curl -s -X POST localhost:9901/drain_listeners?inboundonly
curl -s -X POST localhost:9901/healthcheck/fail

The drain_listeners command stops the listener from accepting new connections, while healthcheck/fail marks Envoy unhealthy in the orchestrator. After in-flight requests finish, the pod can be stopped without losing traffic — a pattern that must be used on every rolling update.

#Multi-Zone and Multi-Cluster

#Prioritized Local Zones

In a multi-zone deployment, Envoy should prefer endpoints in the same zone:

clusters:
  - name: orders_service
    connect_timeout: 0.25s
    type: EDS
    lb_policy: LEAST_REQUEST
    locality_lb_endpoints:
      - priority: 0
        locality:
          zone: us-east-1a
        lb_endpoints:
          - endpoint:
              address:
                socket_address:
                  address: 10.0.1.10
                  port_value: 8080
      - priority: 1
        locality:
          zone: us-east-1b
        lb_endpoints:
          - endpoint:
              address:
                socket_address:
                  address: 10.0.2.10
                  port_value: 8080

locality_lb_endpoints with priority directs Envoy to use the endpoints in its own zone first, then other zones when the first isn't available. This reduces cross-zone latency and bandwidth costs.

#Health and Load Configuration

The recommended pattern combination for multi-cluster:

• Use EDS so endpoints can change dynamically across clusters.
• Set locality_lb_endpoints per zone with priorities.
• Enable health checks and outlier detection per cluster.
• Let the cross-cluster control plane manage aggregate endpoints.

#Scaling the Gateway with HPA

An Envoy gateway on Kubernetes can scale automatically:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: edge-gateway
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: edge-gateway
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

The edge-gateway HPA adds replicas when average CPU goes above 70 percent. Because Envoy is stateless and config comes from the control plane, a new pod becomes useful within seconds.

#Closing

Episode 17 brought Envoy to platform scale: sidecar, gateway, and standalone deployment patterns, high availability of the xDS control plane, zone priorities for latency, and gateway autoscaling.

Key takeaways:

• Envoy is stateless; scaling means adding instances, not replicating state.
• Sidecar per workload, gateway at the edge, standalone for special tasks.
• The xDS control plane must have more than one instance behind a load balancer.
• Envoy keeps running on the last config if the control plane goes down.
• drain_listeners and healthcheck/fail are the safe shutdown protocol.
• locality_lb_endpoints with priorities makes Envoy prefer local zones.

In the next episode, episode 18, we'll discuss advanced routing and traffic shaping — weighted clusters, mirror traffic and shadowing, header-based routing and path rewrites, plus fault injection for chaos engineering.