Learn Karpenter - Advanced Features

#Introduction

In episode 16 you successfully migrated your cluster from Cluster Autoscaler and understood the differences in its behavior. But Karpenter is not only an autoscaler for standard EC2 nodes. Some workloads need dedicated machines — bare metal for licensing and full control, or on-premises capacity that has already been purchased and cannot be ignored. There are also organizations that are not on AWS at all.

This episode opens Karpenter's advanced features from two directions. First, static capacity and special nodes: how NodeClass manages bare metal and on-premises nodes. Second, the cross-platform ecosystem: Karpenter on AKS, EKS Anywhere, and the role of community providers that extend Karpenter beyond AWS.

#Static Capacity: Static and Custom Nodes

So far we have discussed nodes that are launched on demand. But there are scenarios where capacity already exists physically: bare metal purchased for licensing, or on-premises servers already in place. Karpenter supports these scenarios by treating static nodes as part of the NodePool ecosystem.

The key concept: Karpenter does not launch instances for static capacity, but it still manages their scheduling lifecycle. Static nodes are registered with the same labels and taints as other nodes, so Karpenter can schedule workloads there and honor the taints they carry.

#Bare Metal on AWS

AWS provides bare metal instances such as m5.metal and c6i.metal for workloads that demand direct hardware access, per-socket licensing, or specific virtualization needs. Metal instances also unlock features like SR-IOV and NMI control that are not available on virtual instances.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: metal
spec:
  template:
    spec:
      taints:
        - key: workload-type
          value: metal
          effect: NoSchedule
      requirements:
        - key: node.kubernetes.io/instance-type
          operator: In
          values: ["m5.metal", "c6i.metal"]
  disruption:
    consolidationPolicy: WhenUnderutilized

#Managing On-Premises Capacity

For on-premises capacity, Karpenter uses the same mechanism: nodes are registered to the cluster with labels that match NodePool requirements, and Karpenter treats them as available capacity. What differs is who launches the node — not Karpenter, but on-premises tooling such as metal provisioning or OpenStack.

kubectl label node prem-01 \
  node.kubernetes.io/instance-type=on-prem-general \
  topology.kubernetes.io/zone=onprem-a \
  karpenter.sh/registered=true
 
kubectl taint nodes prem-01 workload-type=onprem:NoSchedule

Once the node is registered, create a NodePool with matching requirements and the appropriate tolerations. Karpenter will schedule workloads onto those on-premises nodes alongside cloud nodes, with the preference you define through weight.

#Karpenter on AKS

Karpenter officially supports AKS (Azure Kubernetes Service) since v1.0. The NodePool and NodeClaim concepts are the same, but capacity is built by the Azure provider. AzureNodeClass replaces EC2NodeClass, with settings such as image, userData, and pod identity.

apiVersion: karpenter.azure.com/v1alpha2
kind: AzureNodeClass
metadata:
  name: default
spec:
  image:
    id: /subscriptions/SUBSCRIPTION/resourceGroups/RG/providers/Microsoft.Compute/images/aks-ubuntu
  userData: |
    #cloud-config
    runcmd:
      - echo "node karpenter aks" > /etc/motd

NodePools on AKS add VM size and SKU family choices through Azure requirements, for example keys like kubernetes.azure.com/sku-family. Consolidation, drift, and disruption behavior is the same as on AWS, so the knowledge from previous episodes applies directly.

#EKS Anywhere

EKS Anywhere brings the EKS API and tooling to your own datacenter. Because an EKS Anywhere cluster is still Kubernetes, Karpenter can be installed there. But the cloud provider is not AWS — capacity comes from vSphere or local bare metal, and Karpenter needs a provider that understands those resources.

In an EKS Anywhere environment, NodePools still manage scheduling intent, but node creation depends on the available provider. Best practice: use Karpenter for consolidation and workload scheduling, while letting existing tooling handle physical provisioning.

#Community Providers

The Karpenter ecosystem grows through community providers. These are cloud provider implementations outside the official ones that allow Karpenter to run on other platforms. Karpenter Core is designed so that node creation and maintenance are delegated to the provider, letting the community add support for new platforms.

Before adopting an unofficial provider, run load tests in a separate environment and pay attention to the contract between versions. Karpenter Core changes quickly, and a provider that falls behind can leave your cluster unable to scale until you upgrade.

#Closing

Advanced features extend Karpenter from an EC2 node autoscaler into a cross-platform capacity scheduling framework.

Key takeaways:

• Static capacity is legitimate: bare metal and on-premises nodes can be managed alongside cloud nodes through consistent labels, taints, and tolerations.
• Bare metal needs isolation: NodePool taints and dedicated pod tolerations prevent ordinary workloads from wasting money on metal instances.
• AKS is officially supported: AzureNodeClass and VM size requirements make the same Karpenter patterns work on Azure.
• EKS Anywhere can use Karpenter: EKS clusters in a datacenter can still be scheduled, with physical provisioning handled by local tooling.
• Community providers need caution: check maturity and update cadence before adopting an unofficial provider.

Karpenter is no longer just an autoscaler — it is a complete capacity scheduling foundation. But when everything runs automatically, you will need the ability to figure out why something isn't working. In episode 18 we cover Troubleshooting — reading controller logs, NodeClaim and NodePool conditions, webhook validation, and common problems such as exhausted instance quotas, empty Spot capacity, and pods that never get scheduled. See you there!