Learn Flannel - Production-Ready Architecture

#Introduction

All previous episodes are building blocks. This episode assembles them: how the small decisions in each episode come together into an architecture worthy of production.

Episode 21 guides you through assembling a production Flannel architecture: choosing a backend, tuning the MTU, pairing a policy engine, crafting an upgrade strategy, and preparing operations with a GitOps pipeline, observability, node sizing, and an incident runbook.

#Choosing a Backend for Production

#VXLAN vs host-gw

The first and most important decision: the backend. In production, the choice comes down to two main candidates. VXLAN is flexible and safe for diverse host networks. host-gw is the fastest but demands that nodes can reach each other at the network layer.

net-conf.json: |
  {
    "Network": "10.244.0.0/16",
    "MTU": 1450,
    "Backend": {
      "Type": "vxlan",
      "VNI": 1,
      "Port": 4789
    },
    "TrafficManager": "nftables"
  }

If security of inter-node traffic is a requirement, choose WireGuard. This decision is driven by three factors: host network topology, security requirements, and performance priorities.

#Node and Cluster Sizing

Estimate the number of nodes and Pod churn. Each node consumes one /24 subnet, and all nodes store routes to all other subnets. Clusters of hundreds of nodes can still run, but monitor the API server load as the coordination center for leases.

#Tuning the MTU and Interface

#A Verified MTU

Apply the MTU values already tested in staging. Remember the 50-byte VXLAN overhead and the fragmentation symptoms from episode 16. Verify with a ping -M do before production takes full load.

#An Explicit Interface

Always set the Iface field explicitly. In production, nodes often have many interfaces, and flanneld's guess can be wrong. This single line of configuration saves you from hard-to-diagnose problems.

#Companion Policy Engine

#Security on Top of Flannel

Flannel handles connectivity; the security layer above it is handled by a policy engine. Choose Calico policy-only or Cilium based on what your team is comfortable with. Make sure a default-deny policy is thoroughly tested in staging before production, with an allowlist for DNS, monitoring, and ingress.

#Upgrade Strategy

#Canary and Rollback

Upgrade Flannel with a canary pattern: apply it on one node, verify, then let the rolling update finish the rest. GitOps is the backbone of this strategy:

kubectl set image ds/kube-flannel-ds -n kube-flannel kube-flannel=docker.io/flannel/flannel:v0.28.8
kubectl rollout status ds/kube-flannel-ds -n kube-flannel

The kubectl rollout status command monitors progress. If a node fails, stop and use GitOps to roll back to the previous version.

#CI/CD and Operations

#GitOps Pipeline

Flannel's configuration lives in the infrastructure repository and is applied by Argo CD or Flux. Changes pass through pull requests and reviews. The image version is pinned clearly so the diff is easy to read.

#Observability

From episode 12, apply the minimum: scrape flanneld metrics into Prometheus, set up alerts for excessive flanneld Pod restarts and differences in subnet counts between nodes, and make sure flanneld logs are available centrally.

#Incident Runbook

Prepare a runbook for the most common incidents before they happen:

kubectl get nodes
kubectl get pods -n kube-flannel -o wide
kubectl get lease -n kube-system

These three commands — kubectl get nodes, kubectl get pods -n kube-flannel, and kubectl get lease -n kube-system — are the first gate for every incident. A good runbook writes down the steps, commands, and completion criteria for each scenario.

#Evaluating the Architecture Periodically

#Periodic Audits as a Habit

A production architecture is not static. Every few months, revisit the decisions you've made: is the backend still right, does the MTU still match the network path, does the policy engine still fit the requirements? Periodic audits keep the architecture aligned with reality in the field.

#Documentation and Team Review

Record every architectural change along with its rationale. When new team members join, this documentation becomes a valuable entry point. Involve team review for big decisions, because diverse perspectives catch risks that a single person would miss.

#Conclusion

Episode 21 assembled a production Flannel architecture: backend and MTU decisions, a companion policy engine, a canary upgrade strategy, and operations with GitOps, observability, and a runbook.

Key takeaways:

• The production backend is chosen from topology, security, and performance priorities.
• MTU and interfaces must be verified in staging before production.
• A companion policy engine completes security on top of Flannel.
• Upgrades use a canary pattern with rollback through GitOps.
• Minimal observability: metrics, alerts, and centralized logs.
• A runbook documents tested incident-resolution steps.

In episode 22, the final episode, we will look around us: the alternative ecosystem and final reflection — a thorough comparison of Flannel vs Calico vs Cilium, a recap of the journey from episode 0 to 21, a production-grade Flannel checklist, and Flannel's future in the Kubernetes ecosystem.