Learn GitLab CI/CD - Progressive Delivery (Canary & Blue-Green Deployments)

#Introduction

In episode 16 we successfully deployed to Kubernetes via the GitLab Agent. But the real question appears right after the deploy button is pressed: "What if this new version is broken?" An all-at-once deployment method makes even a small mistake impact every user. Progressive delivery answers this by treating release as a gradual process, not a one-shot moment.

#The Progressive Delivery Concept

The analogy: when an airport builds a new runway, they don't close all the old runways and open the new one at once. The first planes landing on the new runway are test crews and engineers. Once proven safe, certain commercial flights are allowed, then finally all of them. Progressive delivery applies the same logic to software: ship the new version to a small portion of users, observe, then expand.

Progressive delivery covers three common strategies:

Rolling update is already the Kubernetes default. Canary is finer: only part of the traffic. Blue-green keeps two versions alive side by side then moves all traffic at once — fast, but requires double capacity.

#Canary Deployment in GitLab CI

The goal: ship the new version with 10 percent traffic, verify, then raise it to 100 percent. In GitLab, we make this a two-step pipeline separated by a health check.

stages:
  - canary
  - production
 
deploy_canary:
  stage: canary
  image: alpine/helm:latest
  script:
    - helm upgrade --install my-app ./chart
      --set canary.enabled=true
      --set canary.weight=10
      --set image.tag=$CI_COMMIT_SHA
  environment:
    name: production/canary
    url: https://myapp.example.com

This job installs the chart with 10 percent weight routed toward the new pod version. The production/canary environment marks this as a sub-environment of production, so GitLab displays it separately on the Environments page while still connecting it to its parent environment.

#Health Check Before Promotion

After the canary runs, we must not blindly promote it. A verification job checks 5xx metrics via Prometheus. If the error rate is low, promotion continues; if not, the pipeline fails — and this is where automatic rollback comes into play:

error_rate=$(curl -s "http://prometheus.example.com/api/v1/query" \
  --data-urlencode 'query=sum(rate(http_requests_total{status=~"5..",app="my-app"}[5m])) / sum(rate(http_requests_total{app="my-app"}[5m]))' \
  | jq -r '.data.result[0].value[1]')
 
threshold=0.01
if (( $(echo "$error_rate > $threshold" | bc -l) )); then
  helm rollback my-app 1
  echo "Canary bermasalah, rollback ke revisi 1"
  exit 1
fi

This script calculates the 5xx request ratio over the last five minutes. The threshold=0.01 value means 1 percent error is the tolerance limit. If exceeded, helm rollback returns the release to the previous revision and the job exits with an error code — turning the pipeline red as a signal.

#Promotion to 100 Percent

The health check passed, now it's time to scale the canary up to full:

promote_production:
  stage: production
  image: alpine/helm:latest
  script:
    - helm upgrade --install my-app ./chart
      --set canary.weight=100
      --set image.tag=$CI_COMMIT_SHA
  environment:
    name: production
    url: https://myapp.example.com

The weight is changed to 100, all traffic flows to the new version, and the production/canary environment automatically becomes irrelevant. The whole process can be repeated for every release.

#Blue-Green Deployment

If your team prefers instant switching and has spare capacity, blue-green keeps two permanent environments: blue and green. The new version is deployed to the environment currently not serving traffic, tested, then the service is switched:

apiVersion: v1
kind: Service
metadata:
  name: my-app
spec:
  selector:
    app: my-app
    color: green

The pipeline will change the selector color during deploy. Because the selector is only two lines that change, the switch can happen at any time — including rollback, just switch back to blue. The obvious downside: two sets of resources always run, so cluster costs go up.

#Closing

Progressive delivery turns release from a tense event into a measurable process:

• Gradual traffic (canary) or instant switching (blue-green) minimizes the impact of failures.
• The production/canary environment in GitLab marks a canary sub-deploy.
• Prometheus-based health checks hold promotion when the error rate crosses the threshold.
• Automatic rollback via helm restores service without waiting for a human.
• Choose canary as your starting point; blue-green when you need fast switching.

In the next episode 18 we cover Auto DevOps and the CI/CD Component Catalog — how to build "free" pipelines and reusable components across teams. See you there!