Learn MetalLB - IP Planning & Network Scale

#Introduction

External IPs are the scarcest resource on a bare-metal cluster. Every LoadBalancer Service borrows one IP from a pool, and when the pool is empty, new Services will never get an IP — the symptom: an EXTERNAL-IP that never fills in. Episode 17 covers IP planning & network scale: calculating requirements, understanding the ARP and BGP limits, designing large pool strategies, and avoiding IP exhaustion and allocation fragmentation.

Good IP planning is done once and saves years of operations. Bad planning forces you to rebuild pools in the middle of production — an unpleasant experience. Let's do it right from the start.

#Capacity: Number of Services versus Number of IPs

#Calculating Requirements

Every LoadBalancer Service uses one IP from the pool. The basic requirement formula is simple:

• Number of IPs in the pool = maximum simultaneous LoadBalancer Services capacity.
• Add buffer for growth and maintenance.

Example: the pool 192.168.20.200-192.168.20.240 provides 41 IPs, enough for about 41 simultaneous Services. If you're planning 100 Services, you need more than 100 IPs — because maintenance and experiments also use IPs.

#Monitoring Capacity Continuously

Don't guess when a pool is near full — monitor with the metrics from episode 11:

kubectl get svc -A -o wide | wc -l
kubectl get ipaddresspool

kubectl get svc -A -o wide | wc -l gives a sense of the number of Services; compare it with the pool's total capacity. For precise numbers, use the Prometheus metrics metallb_allocator_addresses_in_use_total versus metallb_allocator_addresses_total per pool.

#ARP and BGP Limits

#Layer 2 Limitations

In Layer 2 mode, a single leader node handles all traffic for an IP. The more Services whose leader lands on the same node, the heavier that node's load. Also, every IP is announced via ARP — the more IPs, the more ARP broadcast traffic on the network. At a certain scale, the limit isn't MetalLB but the capability of the node and the switches.

#BGP Limitations

In BGP mode, every non-aggregated IP is advertised as a /32 route. A thousand Services means a thousand routes in the router's routing table. This is still considered reasonable for modern routers, but it carries two risks:

• The routing table bloats as it grows.
• Convergence (route recalculation) slows down when the network changes.

The solution is prefix aggregation (aggregationLength from episodes 8 and 16): with aggregation, hundreds of IPs are advertised as one prefix.

show ip bgp | wc -l

show ip bgp | wc -l on the router shows the number of received route lines. If this number bloats as Services increase, consider more aggressive aggregation or pool separation.

#Large Pool Strategies

#One Big Pool vs Many Small Pools

Two approaches for large scale:

• One big pool: simple, but all Services share one set of IPs without environment separation.
• Many small pools: separates environments, departments, or network segments — far more controlled and easier to audit.

For production, many small pools are almost always better. Per-Service pool selection uses the annotation we covered in episode 9.

#Example Pool Strategy

apiVersion: metallb.io/v1beta2
kind: IPAddressPool
metadata:
  name: apps-pool
  namespace: metallb-system
spec:
  addresses:
    - 10.0.20.0/24
---
apiVersion: metallb.io/v1beta2
kind: IPAddressPool
metadata:
  name: infra-pool
  namespace: metallb-system
spec:
  addresses:
    - 10.0.30.0/24

apps-pool and infra-pool separate application IPs from infrastructure IPs. Function-to-subnet mappings like this make IP allocation easy to predict just by looking at the address — very helpful when troubleshooting.

#Avoiding IP Exhaustion and Fragmentation

#Allocation Fragmentation

Released IPs (after a Service is deleted) can leave holes in the pool. If the controller always allocates the first free IP, a pool with many allocations and de-allocations can become fragmented — making patterns hard to recognize or efficient aggregation hard to apply.

Mitigation: assign IP blocks per function with discipline, avoid mixing many functions in one pool, and periodically monitor usage to spot strange patterns.

#Preventing IP Exhaustion

IP exhaustion isn't a sudden crisis — it's usually predictable. Ways to prevent it:

• Monitor metallb_allocator_addresses_in_use_total per pool with alerts at a certain threshold, for example 80 percent.
• Run regular capacity reviews against your Service growth plan.
• Prepare a backup pool on an unused subnet before the main pool fills up.
• Consider externalTrafficPolicy and Ingress (episode 12) to reduce per-application IP usage.

#Recommended Capacity

As a starting guideline: prepare a pool with a buffer of at least 20 percent above current needs, and plan additional subnets before the main pool hits 80 percent usage. These numbers give you time to plan without waiting for a crisis.

#Conclusion

Episode 17 completes IP planning & network scale: calculating capacity, understanding the ARP and BGP limits, designing a many-pool strategy, and preventing IP exhaustion and allocation fragmentation.

Key takeaways:

• One LoadBalancer Service uses one IP — calculate capacity from the number of Services.
• The Layer 2 limit is the leader node's load and ARP traffic.
• The BGP limit is the routing table size; reduce it with prefix aggregation.
• Many small pools are easier to manage than one big pool.
• Alert at 80 percent pool usage to prevent IP exhaustion.
• Monitor metallb_allocator_addresses_in_use_total regularly.

In the next episode, episode 18, we'll discuss GitOps & config as code — managing IPAddressPool, advertisements, and BGPPeer as code in a repository, applying them with Argo CD and Flux, configuration versioning, review flows, and rolling changes out gradually.