Learn Flannel - Observability & Basic Monitoring

#Introduction

A healthy Flannel feels like it doesn't exist: Pods connect, nothing needs attention. But when there's a problem, observability is the lifesaver. Fortunately Flannel provides logs, metrics, and state you can inspect directly.

Episode 12 builds basic observability: how to read flanneld logs, the metrics flanneld exposes, and inspecting network state such as routes, ARP, and FDB.

#Flannel Logs

#Reading flanneld Logs

Every important flanneld event is recorded in the Pod logs. From startup, to taking a lease, to route synchronization, it's all here:

kubectl logs -n kube-flannel -l k8s-app=flannel -f

The kubectl logs -n kube-flannel -l k8s-app=flannel command shows the logs of all flanneld instances at once. For a single node, add the Pod name flag:

kubectl logs -n kube-flannel <nama-pod> --tail=50

#Log Patterns to Watch Out For

Log lines with the words error or failed are the first alarm. Examples: failed to acquire subnet lease, or unable to contact API server. Both patterns point directly to the datastore or permissions, not to the network itself.

#flanneld Metrics

#Subnet Allocations Metric

flanneld exposes Prometheus metrics on port 9091 inside the Pod. The most useful metric is flannel_subnets_total, which shows how many subnets that node knows.

kubectl exec -n kube-flannel -l k8s-app=flannel -- wget -qO- localhost:9091/metrics | grep flannel_subnets

The output of the command above shows metric values you can use to verify synchronization: each node should see the same number of subnets as the number of nodes.

#Healthcheck

Besides metrics, flanneld has a healthcheck endpoint that can be used as a probe. Combining metrics and logs gives enough health signals to monitor the network without extra tooling.

#Inspecting Routes, VXLAN, ARP, and FDB

#The Route Table

The most direct way to verify the network: check the routes pointing to other nodes' subnets:

ip route | grep flannel

The output of ip route | grep flannel shows one route per other node. If one subnet is missing, that node's lease synchronization has a problem.

#The VXLAN Interface

Inspect the interface details to make sure of the state and MTU:

ip -d link show flannel.1

The state and MTU columns in ip -d link show flannel.1 tell you whether the interface is up and whether the MTU matches the calculation.

#ARP and FDB

These two tables determine where packets get wrapped:

ip neigh show dev flannel.1
bridge fdb show dev flannel.1

The ip neigh show dev flannel.1 command shows MAC neighbors, and bridge fdb show dev flannel.1 shows the mappings to host IPs. If one of them is empty while other nodes exist, synchronization hasn't finished.

#Setting Up Prometheus Scraping

#ServiceMonitor for Flannel

For production, integrate flanneld metrics into Prometheus. Create a Service pointing to port 9091 on each flanneld Pod, then define a ServiceMonitor or scrape config:

apiVersion: v1
kind: Service
metadata:
  name: kube-flannel-metrics
  namespace: kube-flannel
spec:
  selector:
    k8s-app: flannel
  ports:
    - name: metrics
      port: 9091

Notice the k8s-app: flannel selector, which matches the DaemonSet Pod labels. After the Service is created, Prometheus can be pointed to scrape the metrics port.

#Alerts Worth Installing

Several conditions deserve an alert: kube-flannel Pods restarting too often, the flannel_subnets_total metric differing between nodes, and the number of nodes not matching the subnets seen. These simple alerts catch the majority of Flannel network failures before users report them.

#Conclusion

Episode 12 gave you Flannel's basic observability toolkit: logs for reading events, metrics for monitoring health, and route, ARP, and FDB inspection for direct verification.

Key takeaways:

• flanneld logs record every event from leases to synchronization.
• The flannel_subnets_total metric shows how many subnets a node knows.
• The healthcheck endpoint and port 9091 can be used as Prometheus probes.
• Flannel routes show other nodes' synchronized subnets.
• ARP and FDB determine the encapsulation destination of VXLAN packets.
• Simple metric-based alerts catch failures before users notice.

In the next episode, episode 13, we will secure Flannel: security and hardening — understanding Flannel's security model that focuses on connectivity, and enabling the WireGuard and IPsec backends to encrypt traffic between nodes.