Learn MetalLB - Production-Ready Architecture

#Introduction

After 20 episodes of theory, hands-on practice, and troubleshooting, it's time to tie everything together into one whole: a production-ready MetalLB architecture. Episode 21 isn't about a single new feature — it's about design decisions: pool planning, mode selection, router integration, high availability, upgrade strategy, GitOps pipeline, observability, sizing, and an incident runbook.

A good production architecture doesn't emerge from one "correct" configuration. It emerges from decisions made with an understanding of trade-offs — exactly what you've learned throughout this series. This episode is the complete blueprint.

#IP Pool Planning

#Choosing the Block and Structure

Pool planning is the first step that shouldn't be rushed. Recommendations for production:

• Choose an IP block that is separate from the infrastructure subnet.
• Provide a buffer of at least 20 percent above current demand.
• Separate pools per environment and function (episodes 9 and 17).
• Document the subnet-to-function mapping in the configuration.

Example production configuration:

apiVersion: metallb.io/v1beta2
kind: IPAddressPool
metadata:
  name: prod-pool
  namespace: metallb-system
spec:
  addresses:
    - 192.168.20.0/24
  avoidBuggyIPs: true

192.168.20.0/24 provides more than 200 usable IPs — enough for most initial needs, with avoidBuggyIPs: true avoiding problem addresses.

#Choosing the Mode: Layer 2 vs BGP

#A Decision Based on Requirements

Choosing the mode is the biggest architectural decision. The guidance:

• Choose Layer 2 if the network is simple, there's no BGP router, and per-Service throughput isn't very high.
• Choose BGP if the cluster has many nodes, needs ECMP to spread traffic, and the network team is ready to manage peering.

kubectl get l2advertisement
kubectl get bgpadvertisement
kubectl get bgppeer

kubectl get l2advertisement, kubectl get bgpadvertisement, and kubectl get bgppeer show which modes are active. A cluster can run both at the same time — one pool announced over L2, another over BGP.

#Router Integration and High Availability

#The Router as Part of the Design

For BGP mode, router integration is a joint project with the network team. Make sure:

• Dual peering to at least two routers (episode 13).
• An explicit routerID for each node.
• Reasonable hold time — not too aggressive.
• The advertised prefixes match the network team's policy.

#Comprehensive High Availability

MetalLB high availability only means anything if the layers beneath it are also HA:

• At least two nodes ready to announce.
• More than one controller replica for allocation availability.
• externalTrafficPolicy: Local with pod replicas spread out (episode 9).
• Scheduled failover simulations (episode 15).

#Upgrade Strategy and GitOps Pipeline

#Upgrade Strategy

A safe upgrade strategy for production:

• All configuration is managed via GitOps (episode 18).
• The upgrade is done in staging first and monitored for a few days.
• Verify resource and Service status after the upgrade.
• Prepare rollback with version control.

#GitOps Pipeline

Argo CD or Flux applies configuration automatically from the repository. Recommended practices:

• One repository per environment, or one repository with overlays.
• CI runs YAML validation before merge.
• Role separation: who is allowed to change pools, peers, and advertisements.

#Observability and Sizing

#Observability Stack

Monitor MetalLB with the stack you built in episode 11:

• Prometheus for metrics and alerts.
• Grafana dashboards for visualization.
• Important alerts: pool usage above 80 percent, BGP sessions down, fewer speakers.

kubectl get pods -n metallb-system
kubectl top pods -n metallb-system

kubectl top pods -n metallb-system shows CPU and memory usage. This data is the basis for sizing: make sure the resource requests for controller and speaker match the load, especially on clusters with many Services and nodes.

#Realistic Sizing

As a starting point: the controller doesn't need many resources — it only processes allocations. The speaker shares the load per node. If metrics show throttling or OOM, raise the resource limits — but measure first, don't guess.

#Incident Runbook

#A Document That Keeps Operations Calm

An incident runbook is a document that explains concrete steps when a problem occurs. For MetalLB, the runbook covers:

• Common symptoms and first diagnostic steps (episode 19).
• Contacts to reach out to — platform and network engineers.
• Temporary and permanent mitigation steps.
• Configuration rollback procedures.

kubectl get svc -A -o wide
kubectl get events --all-namespaces | grep -i metallb

kubectl get events --all-namespaces | grep -i metallb is the opening step almost always written into a runbook — it's fast and often reveals the root cause immediately. Practice this runbook in incident simulations so the team gets familiar with it.

#Conclusion

Episode 21 completes the production-ready architecture: pool planning, mode selection, router integration, high availability, upgrade strategy, GitOps pipeline, observability, sizing, and an incident runbook — all assembled into one cohesive architecture.

Key takeaways:

• Production pools are separated per environment with capacity buffer.
• Layer 2 for simple networks; BGP for scale and ECMP.
• Dual peering, explicit routerID, and reasonable hold time for BGP HA.
• Upgrades always go through staging and the GitOps pipeline.
• Alerts for pool usage, BGP sessions, and speaker count.
• The incident runbook spells out symptoms, diagnosis, mitigation, and rollback.

In the next episode — the final one, episode 22 — we'll discuss the alternative ecosystem & final reflection: a comprehensive comparison of MetalLB versus kube-vip, cloud LBs, and NodePort, when to choose each, a recap of the journey from episodes 0 to 21, a production-grade MetalLB checklist, and the future direction of MetalLB in the Kubernetes ecosystem.