Learn Karpenter - Migration from Cluster Autoscaler

#Introduction

In episode 15 you put together capacity best practices and FinOps for Karpenter. But you may be wondering: what if the cluster is still running Cluster Autoscaler (CA)? Replacing the node autoscaler in a production cluster feels intimidating — CA has been running for a long time, workloads have adapted to its behavior, and one mistake can leave the whole cluster stuck without new nodes.

This episode covers migrating from Cluster Autoscaler to Karpenter systematically. You will learn when to replace CA, run both side by side, convert nodegroups into NodePools, understand the differences in scheduling behavior, and manage downtime during the rollout.

#When to Replace Cluster Autoscaler

Cluster Autoscaler is node-group based: it only adds or removes instances within a predefined group. Karpenter works directly at the instance level, so it is more flexible. Consider migrating when:

• Speed matters: CA takes minutes to scale up because it waits for the node group; Karpenter launches instances in seconds.
• Cost efficiency: CA does not consolidate aggressively; Karpenter packs workloads together and makes extensive use of Spot.
• Instance customization: CA is limited to the instance types in a launch template; Karpenter chooses from hundreds of types.
• Management overhead: node groups are separate AWS resources that must be maintained; NodePools and NodeClasses are Kubernetes-native.

#Gradual Migration and Coexistence Strategy

Karpenter and Cluster Autoscaler can coexist in the same cluster as long as one condition is met: they must not manage the same node group. CA only moves the number of instances within a specific node group, while Karpenter creates its own nodes. If both touch the same node group, a scaling war breaks out: CA scales up, Karpenter scales down.

kubectl scale deployment cluster-autoscaler \
  -n kube-system \
  --replicas=0

A safe gradual strategy:

1. Install Karpenter without removing CA.
2. Create a NodePool with a dedicated taint that the old node groups do not use.
3. Move part of the workload to Karpenter nodes through tolerations and scheduling.
4. Once the workload is stable, turn off CA and let the old node groups shrink.
5. Delete the node groups and remove CA entirely.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: migration
spec:
  template:
    spec:
      taints:
        - key: karpenter.sh/migrated
          effect: NoSchedule
  disruption:
    consolidationPolicy: WhenUnderutilized

Pods that are moved add the karpenter.sh/migrated toleration so they are scheduled onto Karpenter nodes, while other workloads keep running on the old node groups without disruption.

#Converting a Nodegroup into a NodePool

Existing node groups are not automatically managed by Karpenter. You must express the same intent as a NodePool and a NodeClass. The conversion map is simple: the instance type intent of a node group becomes requirements, and the launch template becomes a NodeClass.

Note that Karpenter does not know an explicit max size per NodePool. Capacity limits are enforced through a combination of limits on resources, disruption budgets, or tagging policies. Adjust team expectations: this behavior is more flexible but requires disciplined control.

#Differences in Scheduling Behavior

Karpenter and CA differ fundamentally in how they view capacity. CA asks "are there pods that cannot be scheduled, and can the node group grow?", while Karpenter runs a binpacking simulation against instance types before deciding to create a node.

# Node dengan label managed oleh Karpenter
kubectl get nodes -l karpenter.sh/nodepool
 
# Node yang berasal dari node group eksisting
kubectl get nodes -l eks.amazonaws.com/nodegroup

These differences have a real impact: Karpenter packs pods more tightly, so fewer nodes are used; it selects instances that match pod needs exactly, and it does not wait for node group availability. As a result, applications that rely on deterministic placement must be verified again.

#cluster-autoscaler.kubernetes.io Annotations

Workloads written for CA often carry the annotation cluster-autoscaler.kubernetes.io/safe-to-evict: "false" to prevent eviction during scale-down. Karpenter does not read this annotation. Instead, use a Pod Disruption Budget to govern pods that must not be disturbed during consolidation.

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: api-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: api

#Managing Downtime

A well-planned migration does not require downtime. The key is having new nodes available before removing old ones: make sure the Karpenter NodePool is active and new workloads are ready, and only then turn off CA and shrink the node groups.

The most sensitive point is the difference in terminationGracePeriodSeconds and drain behavior. CA and Karpenter both drain, but the order and priority differ. Test on a staging environment with production-like load, and prepare a rollback by keeping the original CA configuration.

#Closing

Migrating from Cluster Autoscaler is a project that can go smoothly if done gradually and with the right protection.

Key takeaways:

• Migrate when the value is clear: speed, cost efficiency, or instance flexibility are valid reasons; trends are not.
• Coexistence is possible: Karpenter and CA can run together as long as they do not manage the same node group.
• Convert concepts deliberately: node groups become NodePools, launch templates become NodeClasses, and max size becomes budget control.
• Replace CA annotations with PDBs: cluster-autoscaler.kubernetes.io/safe-to-evict is not read by Karpenter, so audit and convert it before consolidation becomes active.
• A rollback is always available: keep a small node group around for a few release cycles after migrating as a safety net.

The migration is done, and Karpenter now manages the primary capacity. But there are many advanced features still unexplored. In episode 17 we cover Advanced Features — static capacity and dedicated nodes such as bare metal and on-premises via NodeClass, then Karpenter on other platforms such as AKS, EKS Anywhere, and community providers. See you there!