Learn Karpenter - Provisioning & Binpacking

#Introduction

In episode 5 you got to know EC2NodeClass and how this object determines the infrastructure side of nodes launched by Karpenter: which AMI is used, which security group and subnet, down to the instance profile for IAM. In this episode 6 we step up a level: we discuss how Karpenter decides which instance type to create, and why that decision often feels magical when it's actually the result of a very deterministic process called binpacking.

Once this episode is done, you'll understand three things: what binpacking is and how Karpenter groups pods, how to adjust the NodePool's spec.template to influence provisioning results, and the difference between Spot and On-Demand capacity via the karpenter.sh/capacity-type label.

#Binpacking: Choosing the Smallest Sufficient Instance

#The Problem Behind Provisioning

Kubernetes nodes are not cheap. Every node launched, whether on-demand or spot, still has to be paid for. The larger the instance type used for a small load, the more money wasted. Cluster Autoscaler generally works at the node group level, so instance type choices are locked in from the start. Karpenter is different because it chooses instance types granularly for each batch of pods.

#How Karpenter Groups Pods

When there are pods that can't be scheduled onto existing nodes, Karpenter doesn't launch nodes one by one. The Karpenter controller gathers a group of unscheduled pods within a short time window, then calculates their aggregate needs: total CPU, memory, and other resources. From this aggregate need, Karpenter looks for the smallest instance type capable of holding the entire group.

kubectl get pods -n workshop --field-selector=status.phase=Pending
kubectl get nodeclaims

This collection of waiting pods is called a batch. The better those pods are packed, the fewer nodes are needed, and the smaller the total instance footprint launched.

#Customizing spec.template

The aggregate pod need is only one input. Another input comes from the NodePool's spec.template. The template section defines the constraints on the nodes that may be created: which labels must exist, what taints are applied, down to instance type requirements. Karpenter combines the pod need with these template constraints to narrow the candidate instance types.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: default
spec:
  template:
    metadata:
      labels:
        karpenter.sh/capacity-type: spot
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: node.kubernetes.io/instance-type
          operator: In
          values: ["m5.large", "m5.xlarge", "m5.2xlarge"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default

Here karpenter.sh/capacity-type and node.kubernetes.io/instance-type are the two most frequently used requirements. Both become the reference for binpacking.

#Capacity Types: Spot vs On-Demand

#Getting to Know the karpenter.sh/capacity-type Label

Karpenter introduced the karpenter.sh/capacity-type label to distinguish two capacity classes on AWS: spot and on-demand. When Karpenter launches a node, this label is attached to the node so you can see its capacity type directly from kubectl.

On-demand provides stable capacity that isn't terminated mid-way, but is more expensive. Spot uses EC2's unused capacity at a big discount, but can be reclaimed by AWS at any time with a two-minute notice.

#Fallback and Priority Between Types

Karpenter handles Spot and On-Demand in a single NodePool via the value list on the karpenter.sh/capacity-type requirement. The order in the list determines priority: the leftmost value is considered preferred, and the rightmost value serves as backup.

spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]

The configuration above means: if Spot capacity is available, Karpenter will choose Spot. When Spot is scarce in the target availability zone, Karpenter automatically falls back to on-demand so pods keep running without manual intervention.

#Forcing a Specific Capacity Type from the Pod

Sometimes the capacity decision isn't in the NodePool's hands, but in the workload's own. You can force a pod to only be scheduled to a certain capacity with nodeSelector.

apiVersion: v1
kind: Pod
metadata:
  name: stateful-worker
spec:
  nodeSelector:
    karpenter.sh/capacity-type: on-demand
  containers:
    - name: app
      image: nginx

With a nodeSelector like the above, the pod will never be scheduled to a Spot node because the label Karpenter attaches doesn't match.

#Practice: Observing Binpacking Decisions

To see binpacking decisions for real, create a deployment with a small load and observe the instance type Karpenter chooses.

kubectl create deployment web --image=nginx --replicas=2
kubectl get nodes -l karpenter.sh/managed=true -o wide

Look at the instance type column in the output. If the NodePool allows several sizes, Karpenter should pick the smallest one sufficient for those two replicas.

#Common Mistakes

#Trusting the Cheapest Instance Type Too Much

A cheap hourly price isn't necessarily efficient for your workload. An instance with lots of CPU but little memory could make Karpenter launch an extra node just because memory ran out first. Pick candidate instance types whose proportions match the workload.

#Forgetting to Set a Fallback

A NodePool that only lists one capacity type will struggle when that capacity is scarce. Always provide at least two values, or prepare a backup NodePool.

#Closing

This episode showed why binpacking is the heart of Karpenter's efficiency: a group of pods is packed into the smallest sufficient instance, keeping node count and cost minimal.

Key takeaways:

• Binpacking sets the price — Karpenter always picks the smallest instance type that fits a group of pods.
• spec.template is a filter — template requirements narrow instance candidates without changing binpacking logic.
• Capacity type is a label — karpenter.sh/capacity-type distinguishes spot and on-demand, and its ordering sets priority.
• Fallback prevents stuck pods — unavailable Spot automatically switches to on-demand when the values list allows it.

In episode 7, you'll learn how Karpenter keeps a cluster healthy and efficient over the long term via disruption budgets and consolidation — the mechanisms that remove nodes no longer needed and merge workloads onto more efficient nodes. See you there!