Learn Multigress - Canary Deployments & Progressive Delivery

#Introduction

Releasing a new version without disrupting users is one of the biggest challenges in operations. Episode 9 covers canary deployments and progressive delivery with Multigress: shifting traffic slowly with weights, monitoring release health in real time, and automating the process with Argo Rollouts or Flagger.

A canary is the bridge between "release directly" and "manual rollback". Traffic flows in gradually, and the decision to proceed or retreat is made based on data, not luck.

#Traffic Shifting and Canary Release with HTTPRoute Weights

#Basic Canary Structure

The simplest canary: two Services pointing to two Deployments that use different images, with weights in the HTTPRoute determining the proportion:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: api-route
spec:
  parentRefs:
    - name: main-gateway
      namespace: multigress-system
  hostnames:
    - "api.example.com"
  rules:
    - backendRefs:
        - name: api-stable
          port: 8080
          weight: 95
        - name: api-canary
          port: 8080
          weight: 5

Five percent of traffic is already going to the new version. When you're confident, change the weights to 50:50, then to 0:100.

#Changing Weights Progressively

Shift traffic gradually with a patch:

kubectl patch httproute api-route --type=json \
  -p='[{"op":"replace","path":"/spec/rules/0/backendRefs/1/weight","value":25}]'

The kubectl patch httproute api-route --type=json command changes the canary weight to 25 without rewriting the entire file. This process can be repeated on a schedule: 5, 25, 50, 100.

#Header-Based Canary

To test only for an internal team, combine it with a header rule like episode 8. Weights are then used as a second layer for public release.

#Observability Feedback Loops for Deployment Safety

#Metrics That Decide Continue or Retreat

A safe canary requires observation. The metrics you should monitor while a canary runs:

• Error rate (5xx) per route.
• p95 and p99 latency.
• Request rate reaching the canary backend.

Prometheus query to compare the two backends:

sum(rate(multigress_http_requests_total{backend="api-canary"}[5m])) by (code)
sum(rate(multigress_http_requests_total{backend="api-stable"}[5m])) by (code)

Compare the status code distribution of both backends. If api-canary produces far more 5xx than api-stable, stop the canary immediately.

#Safe Thresholds in Practice

Set thresholds before the release, for example:

• The canary error rate must not exceed stable plus one percent.
• The canary p95 latency must not exceed 1.2 times stable.

Without thresholds, observability is just a pretty screen with no decisions. A feedback loop is only useful when there are explicit rules for acting.

#Integration with Argo Rollouts or Flagger

#Argo Rollouts with Multigress

Argo Rollouts can automate weight increases and analysis. Analysis is done via an AnalysisTemplate:

apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: canary-error-rate
spec:
  metrics:
    - name: error-rate
      interval: 30s
      successCondition: result < 1
      provider:
        prometheus:
          address: http://kube-prometheus-stack-prometheus.monitoring:9090
          query: |
            sum(rate(multigress_http_requests_total{backend="api-canary",code=~"5.."}[2m]))
            / sum(rate(multigress_http_requests_total{backend="api-canary"}[2m]))

The template above triggers a measurement every 30 seconds. If the canary error rate exceeds one percent, the analysis fails and the Rollout automatically returns to stable.

#Rollout That Uses Multigress Weights

The Rollout then uses the Multigress HTTPRoute weights as canary steps:

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: api
spec:
  strategy:
    canary:
      canaryService: api-canary
      stableService: api-stable
      trafficRouting:
        multigress:
          httpRoute: api-route
      steps:
        - setWeight: 10
        - pause: { duration: 10m }
        - setWeight: 50
        - pause: { duration: 10m }
        - setWeight: 100

The flow above holds traffic at 10 percent for 10 minutes, then 50 percent, then full. The kubectl argo rollouts get rollout api command shows analysis status and canary steps in real time.

#Flagger as an Alternative

Flagger works on a similar principle but uses a regular Kubernetes Deployment and handles its own analysis. The choice depends on your team's ecosystem: Argo Rollouts integrates with Argo CD, while Flagger is lighter and integrates with Prometheus and Grafana.

#Closing

Episode 9 closed the progressive delivery loop: you know how to shift traffic with weights, set success thresholds from observability, and automate the whole process with Argo Rollouts or Flagger.

The key takeaways:

• Weight on HTTPRoute shifts traffic gradually and reversibly.
• Change weights progressively with kubectl patch, not by editing the whole file.
• A feedback loop needs explicit thresholds: maximum error rate and latency.
• Argo Rollouts and Flagger automate analysis and weight shifting.
• Rolling back a canary is as easy as returning the weight to 100 percent stable.

In the next episode 10 we'll discuss multi-tenant routing & namespace isolation — domain-based routing across namespaces, gateway scoping, namespace isolation patterns, and managing shared gateways for many teams. The canary you built will live inside a new multi-tenant structure.