Learn Calico - Multi-Cluster & Federation

#Introduction

One cluster is not always enough. Large organizations run many clusters: one per region, one per team, or one per environment. Episode 16 covers how Calico connects and secures those clusters.

Two main capabilities we'll learn: BGP peering between clusters so pods in different clusters can talk to each other directly, and consistent policy management across all clusters without rewriting configuration over and over.

#BGP Peering Between Clusters

#The Basic Concept

Two Calico clusters can be connected with BGP peering. Each cluster uses a different AS number, then one node in each cluster is peered with the other cluster. As a result, pod routes from cluster A are advertised to cluster B and vice versa.

Cluster A (AS 64510)              Cluster B (AS 64511)
   node-a1  <--- BGPPeer --->  node-b1
   pod CIDR 192.168.0.0/16        pod CIDR 10.244.0.0/16

Once the peering is Established, pods in cluster A can reach pods in cluster B using their pod IPs directly — no cross-cluster Service and no inter-cluster tunnel required.

#Configuring Inter-Cluster BGPPeer

In cluster A, create a BGPPeer pointing at a node in cluster B:

apiVersion: projectcalico.org/v3
kind: BGPPeer
metadata:
  name: peer-cluster-b
spec:
  nodeSelector: hostname == 'node-a1'
  peerIP: 198.51.100.7
  asNumber: 64511

peerIP is the IP of the node to be peered with in cluster B, and asNumber is cluster B's AS. In cluster B, create a symmetric BGPPeer pointing at a node in cluster A.

#Verifying the Peering

calicoctl node status | grep -A4 peer-cluster-b
calicoctl get bgppeer
kubectl exec -n calico-system ds/calico-node -- ip route | grep 10.244

calicoctl node status shows the inter-cluster peering state, and a route to 10.244.0.0/16 (cluster B's CIDR) on a cluster A node proves the advertising is working.

#Cross-Cluster Security Model

#Policy Does Not Cross Cluster Boundaries

It is important to understand: Calico NetworkPolicy works per cluster. A GlobalNetworkPolicy in cluster A is not applied in cluster B. Cross-cluster security is built with policies in each cluster that select traffic coming from the other cluster.

#Labeling Traffic from Another Cluster

A common pattern: each cluster labels the node that accepts external peering, then policy restricts what is allowed in. The example below, in cluster A, limits incoming traffic from cluster B to specific pods only:

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: security-allow-intercluster
spec:
  tier: security
  order: 90
  selector: app == 'gateway'
  types:
    - Ingress
  ingress:
    - action: Allow
      protocol: TCP
      source:
        nets:
          - 10.244.0.0/16
      destination:
        ports:
          - 443

The source nets use cluster B's pod CIDR, and the destination is limited to the gateway workload on port 443. All other cross-cluster traffic is rejected by default deny.

#Consistent Global Policy Management

#Shared Templates and Repository

Because policy does not propagate automatically between clusters, consistency is maintained through a single source of truth: a repository containing policy templates. One template change is applied to all clusters through a CI pipeline. This is the bridge toward the GitOps approach we explore in depth in episode 18.

An example of a simple distributed template:

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: platform-default-deny
spec:
  tier: platform
  order: 100
  selector: all()
  types:
    - Ingress
    - Egress
  ingress: []
  egress: []

#Running Policy on Many Clusters

With kubectl contexts, one command can be repeated across many clusters:

for ctx in prod-asea prod-eu staging-asea; do
  kubectl --context "$ctx" apply -f baseline/
done
kubectl --context prod-asea get globalnetworkpolicy -o wide
kubectl --context prod-eu get globalnetworkpolicy -o wide

The for ctx in ... loop runs the apply against each context, and the kubectl --context commands verify the results one by one.

#Avoiding Multi-Cluster Pitfalls

#IPPool Overlap

The most common pitfall: two clusters using the same pod CIDR. When BGP is peered, routes collide and routing breaks. Always make sure each cluster uses a unique CIDR.

#Kube-proxy and Cross-Cluster

Remember: Kubernetes Services do not automatically span clusters. Cross-cluster access happens at the pod level (direct pod IPs) or through Ingress/Service Mesh that connects clusters. Don't expect a Service name from cluster A to resolve from cluster B.

#Full Verification

calicoctl get nodes -o wide
calicoctl get ippool -o wide
kubectl exec -n default client -- \
  wget -T 3 -q -O - http://10.244.0.15:8080

calicoctl get ippool -o wide makes sure the CIDRs across clusters do not overlap before connectivity is tested.

#Wrap-Up

Episode 16 opens the world of multi-cluster: BGP peering between clusters for direct routing, a security model managed per cluster, and a shared source of truth for consistent policy.

Key takeaways:

• BGP peering between clusters enables cross-cluster pod access.
• Every cluster uses a unique AS number and pod CIDR.
• NetworkPolicy works per cluster; consistency is kept through shared templates.
• Cross-cluster traffic is restricted with policies that select the source CIDR.
• A CI/CD pipeline applies the same templates to all clusters.
• IPPool overlap is pitfall number one to prevent.

Next, in episode 17, we cover advanced eBPF dataplane — kernel prerequisites, advantages like DSR and socket-level policy, the eBPF versus iptables/nftables comparison, and when the right time is to enable it.