Learn KEDA - Best Practice & Cost

#Introduction

In episode 14 we secured KEDA for multi-team use with RBAC, admission webhooks, and policy. Now the most frequently asked question: how much cost can you really save? The answer is in the configuration, not the tool. Two ScaledObjects with identical workloads can differ in cost by 40% simply because of different activationThreshold and cooldownPeriod tuning. This episode covers Best Practice & Cost — FinOps for event-driven workloads and how to keep them reliable as scale changes.

#FinOps: Scale-to-Zero for Batch & AI Inference

Scale-to-zero is the biggest FinOps win with KEDA. Batch workloads and AI inference have a natural profile: very busy, then idle for a long time. Keeping 10 GPUs or 20 CPUs online while the queue is empty is the same as burning money.

Reference: CNCF graduated project, stable version v2.20.2 (July 2026). A minReplicaCount: 0 configuration makes replicas drop to zero when there are no events. Here's a ScaledObject for a Redis-list-based inference worker:

scaledobject-inference.yaml

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: inference-worker
  namespace: ml
spec:
  scaleTargetRef:
    name: inference-worker
  pollingInterval: 15
  cooldownPeriod: 300
  minReplicaCount: 0
  maxReplicaCount: 20
  triggers:
    - type: redis
      metadata:
        address: redis.ml.svc:6379
        listName: inference-jobs
        listLength: "10"

#How Much Can You Save?

Those 40-60% figures aren't empty claims — the percentages equal the proportion of idle time you managed to cut. AI inference that's only busy 4 hours a day and scaled to zero outside of that saves around 60% of compute costs, because CPU/GPU is only paid for while working.

#Activation & Avoiding Thrashing

activationThreshold is the key to keeping KEDA from turning on replicas because of noise. Without activation, a single message can trigger scale-up — replicas come up, no more messages arrive, and they go back down. Repeating up-and-down cycles like this are called thrashing.

In the SQS example below, activationThreshold: 5 means replicas are only turned on when more than 5 messages are queued — filtering out temporary spikes:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: sqs-worker
  namespace: orders
spec:
  scaleTargetRef:
    name: order-worker
  pollingInterval: 15
  cooldownPeriod: 180
  minReplicaCount: 0
  maxReplicaCount: 30
  triggers:
    - type: aws-sqs-queue
      metadata:
        queueURL: https://sqs.ap-southeast-1.amazonaws.com/1234/orders
        queueLength: "10"
      authenticationRef:
        name: sqs-auth

Combine it with a sufficient cooldownPeriod (180-300 seconds): the event-free period before replicas are scaled down. Too short makes the workload scale down in a rush; too long holds idle replicas and wastes money. Tuning both is an iterative activity — monitor it with Grafana.

#Reliability: Fallback & Min Replicas for Critical Paths

Scale-to-zero saves money, but it must come with a safety net. KEDA has fallback: when a scaler fails (for example Prometheus can't be queried or the queue times out), fallback.replicas keeps a minimum replica count without waiting for the scaler to recover.

fallback-config.yaml

spec:
  scaleTargetRef:
    name: order-worker
  minReplicaCount: 0
  maxReplicaCount: 30
  fallback:
    failureThreshold: 3
    replicas: 4
  pollingInterval: 15

What the config above means: if the scaler fails 3 times in a row (failureThreshold: 3), KEDA sets the HPA to 4 replicas (fallback.replicas: 4) and keeps retrying. This is the important difference between reliability and resilience: the scaler may be down, but the service keeps serving.

For critical paths — payment processors, order intake, authentication — don't set minReplicaCount: 0. Cold starting from zero can take 30-60 seconds; that latency is not tolerable. Use a small minReplicaCount (say 2) as a baseline, and let KEDA add replicas on top of it according to events. A little cost for availability is far cheaper than an incident.

#Monitoring Scaler Activation

Nothing is more dangerous than an autoscaler that activates wrong too often — replicas are always on, the bill balloons, and nobody notices. Monitor the metrics KEDA exposes via Prometheus. The KEDA operator exposes keda_scaler_* metrics on port 8080:

kubectl port-forward -n keda deploy/keda-operator 8080:8080
curl localhost:8080/metrics | grep keda_scaler_errors_total
kubectl get scaledobject -n orders

The most useful metrics — do a quick status check with kubectl get scaledobject -n orders and compare against the metric values:

A simple alarm that catches thrashing: keda_scaler_activity changing more than a few times per hour on a single workload, or keda_scaler_errors_total steadily increasing. Also alert when keda_scaledobject_ready stays 0 for more than 5 minutes — a ScaledObject that isn't ready means autoscaling is silently dead.

#Common Mistakes

1. cooldownPeriod too short — the workload keeps going up and down, costs rise because of startup on every cycle.
2. activationThreshold = 0 by default — every message triggers scale-up. Set it above normal noise.
3. Scale-to-zero on a critical path without fallback — one scaler failure becomes a service outage.
4. Not monitoring keda_scaledobject_ready — autoscaling silently breaks, replicas never rise when the queue is full.
5. Forgetting the cold start budget — a large image + GPU scheduling makes pods take a long time; factor it into the responsiveness target.

#Conclusion

This episode connects cost and reliability: scale-to-zero with minReplicaCount: 0 saves 40-60% for batch and AI inference, activationThreshold prevents thrashing, cooldownPeriod maintains stability, and fallback plus min replicas protect critical paths.

Points you should take away:

• Scale-to-zero saves cost equal to the proportion of idle workload time.
• activationThreshold + cooldownPeriod are the two tunings with the biggest cost impact.
• Fallback (failureThreshold + replicas) keeps the service running when a scaler fails.
• Critical paths use a small minReplicaCount, not zero.
• Monitor keda_scaler_* and keda_scaledobject_ready — invisible autoscaling is a time bomb.

Now that cost and reliability are in order, it's time to optimize the infrastructure behind the pods. In the next episode, 16, we discuss Integration with Karpenter: how KEDA scales pods based on events while Karpenter provides nodes in seconds — with combined spot and consolidation cost optimization. See you in episode 16!