Learn KEDA - Performance & Tuning

#Introduction

In episode 18 we learned how to diagnose KEDA when it fails. Now it's time to shift focus: from working to working optimally. Two identical ScaledObjects can have 3x different scale-up latency just because of pollingInterval. This episode covers Performance & Tuning: the parameters to tune, how to calculate replicas that match the load, avoiding provider throttling, and a benchmark methodology so tuning decisions are based on data, not guesses.

#Tuning: pollingInterval vs cooldownPeriod

The two most influential parameters:

• pollingInterval — how often KEDA checks the metric value. Default 30 seconds. Smaller (10-15 seconds) = faster reaction, but more requests to the provider.
• cooldownPeriod — how long KEDA waits without events before scaling replicas down. Default 300 seconds. Larger = calmer up-and-down behavior, but replicas idle longer.

A practical rule of thumb:

#Saturation: Replicas per Message

How many messages does one replica process before saturating? This number determines queueLength (the threshold per replica). Measure it through observation: watch each replica's throughput on the keda_scaler_metrics_value metric, then find the point where adding replicas no longer increases total throughput.

scaledobject-tuned.yaml

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-worker
  namespace: orders
spec:
  scaleTargetRef:
    name: order-worker
  pollingInterval: 15
  cooldownPeriod: 180
  minReplicaCount: 0
  maxReplicaCount: 30
  triggers:
    - type: aws-sqs-queue
      metadata:
        queueURL: https://sqs.ap-southeast-1.amazonaws.com/1234/orders
        queueLength: "25"
      authenticationRef:
        name: sqs-auth

Simple logic: queueLength: 25 means KEDA adds a replica for every 25 messages that pile up. If one replica can process 30 messages/minute, then 25 messages gives a safe buffer without making replicas idle. Lower it to 10 for responsiveness; raise it to 50 for maximum throughput with fewer pods. This number must be validated with real workload benchmarks.

#Provider API Throttling

Each provider limits the number of API requests. Aggressive KEDA polling can hit rate limits — and that makes metrics fail to be read, triggering fallback or Unknown status (episode 18).

A real case: SQS ReceiveMessage is limited per account, Azure Service Bus has per-namespace quotas. KEDA calls the API every pollingInterval for each scaler. Thirty ScaledObjects polling every 5 seconds = 6 requests/second just for polling.

Strategies to deal with it:

curl -s localhost:8080/metrics | grep keda_scaler_errors_total
kubectl logs -n keda deploy/keda-operator | grep -i throttl
kubectl get events -n orders | grep -i throttl

If keda_scaler_errors_total rises and the logs show throttling: raise pollingInterval, reduce the number of scalers per provider, or set minReplicaCount so replicas don't keep touching the API back and forth. For SQS, avoid a queueLength that's too small, which makes replicas scale up and down quickly — every transition triggers additional requests.

#Benchmarking: Measuring Scale-Up Latency

Without measurement, tuning is just opinion — kubectl get scaledobject -n orders -o yaml only shows status, not performance. A reproducible methodology:

1. Define the target. For example: starting from an empty queue, 1000 messages arrive, and the target is all messages processed within 5 minutes.

2. Measure the latency components:

date +%s > /tmp/start     # saat pesan dimasukkan ke queue
kubectl get pods -n orders -w    # catat kapan replica pertama muncul
kubectl wait --for=condition=ready pod -l app=order-worker -n orders --timeout=120s
date +%s > /tmp/end

Total latency = (replica appears − event entered) + (pod ready − replica appears). The first component is pollingInterval + KEDA/HPA time + scheduling; the second is image pull + container startup + readiness probe.

3. Break down the components:

4. Iterate with tests. Repeat with different pollingInterval values and record them in a table. Don't change two variables at once.

#Cold Start & the Cost vs Responsiveness Trade-off

Every tuning decision is a trade-off between cost and response speed. Scale-to-zero saves cost but pays the cold start; minReplicaCount: 0 vs minReplicaCount: 2 is a direct comparison:

comparison-minreplica.yaml

spec:
  minReplicaCount: 0   # hemat biaya, bayar cold start tiap burst
  maxReplicaCount: 30

comparison-warm.yaml

spec:
  minReplicaCount: 2   # selalu siap, biaya idle kecil tapi pasti
  maxReplicaCount: 30

The decision formula: if (burst frequency × cold start) violates your SLO, use a warm minimum. If bursts are rare and the added latency is tolerable, use scale-to-zero. Data from benchmarks determines the optimal number — for example minReplicaCount: 1 might be enough to cut cold start in half because the image is already warm on the node.

#Common Mistakes

1. 5-second pollingInterval for everything. 30 ScaledObjects polling aggressively = provider throttling and logs full of errors.
2. queueLength from a guess. Calculate it from the real throughput of one replica, not a guessed number.
3. Measuring cold start without a warm image. The first number is always worse; average several iterations.
4. Changing polling and cooldown at the same time. You won't know which one contributed to the result.
5. Ignoring keda_scaler_errors_total while tuning. Provider errors silently corrupt your benchmark baseline.

#Conclusion

This episode closes the tuning phase: pollingInterval and cooldownPeriod are the two main knobs, queueLength determines saturation per replica which must be calculated from data, provider throttling is a physical limit to be respected, and benchmarking gives real numbers for scale-up latency and cold start. The cost vs responsiveness trade-off can finally be made from a data table, not intuition.

Points you should take away:

• Small pollingInterval = fast reaction but expensive provider requests.
• cooldownPeriod = the balance between stability and idle replicas.
• queueLength must come from the real throughput of a single replica.
• Provider throttling shows up in keda_scaler_errors_total and operator logs.
• Benchmark one variable per iteration; use data to decide minReplicaCount.

We've just finished the entire core phase — from pre-requisites, operator, scalers, to tuning. In the next episode, 20, we discuss Latest Stable Features (v2.20): the evolution of KEDA v2.x releases, 2026 features, stability fixes, and a glimpse of the v3 roadmap. See you in episode 20!