Learn KEDA - History, Background & Why You Need KEDA

#Introduction

In episode 0 we set up the cluster and installed KEDA correctly. This time we step back from practice and answer the most fundamental question: why does KEDA exist? To understand a tool, the fastest way is to understand the problem the previous tool failed to solve.

That problem is simple but painful: event-driven workloads — workloads that "wake up" because a message arrives — can't be scaled well by Kubernetes' standard autoscaling tools. This episode traces the evolution of autoscaling, the birth of KEDA, and maps its position between pure HPA and the custom-metrics-adapter.

#The Evolution of Autoscaling in Kubernetes

Kubernetes autoscaling basically has two layers: Horizontal Pod Autoscaler (HPA) for the number of pods, and Cluster Autoscaler/Karpenter for the number of nodes. Let's focus on HPA first.

#Two Layers of Autoscaling: Pods and Nodes

It's important to separate the two levels before going further. HPA manages the number of pods in a Deployment based on metrics. Above it sits the second layer: Cluster Autoscaler or Karpenter, which manages the number of nodes — making sure the cluster has capacity to host newly added pods. KEDA works at the first layer (pods); episode 16 will show how KEDA works alongside Karpenter so nodes are available within seconds.

#CPU- and Memory-Based HPA Is Reactive

HPA works with resource metrics: how high a pod's CPU or memory usage is relative to its declared requests. This mechanism is elegant for classic web services, but problematic for event-driven workloads for one main reason: reactive, not proactive.

Imagine an idle queue consumer: its CPU is nearly 0 percent. Then tens of thousands of messages arrive in an instant. The consumer's CPU only starts rising after messages arrive and work begins. That means HPA only reacts once the system is already overloaded — scaling is always one step late. Conversely, a queue can be full while CPU stays low because consumers are waiting on network or I/O responses.

The conclusion: for event-driven workloads, the right signal isn't CPU — it's the number of events waiting. Queue depth, consumer lag, or queue length.

#The Birth of KEDA

In 2019, Microsoft and Red Hat launched a project called KEDA — Kubernetes Event-driven Autoscaling. The idea was revolutionary: instead of replacing HPA, it places an adapter that translates event metrics from various sources into metrics HPA can understand.

Its journey was fast. KEDA joined CNCF as a sandbox project, then was announced as graduated in August 2023 — the highest status for a CNCF project, meaning it's considered mature and production-ready. The v2.x line has been actively developed since 2023 to the present, with the latest stable release being v2.20.2 (July 2026).

Verifying the installed KEDA version

helm list -n keda
kubectl get deploy -n keda
kubectl get crd | grep keda

The output of helm list -n keda will show the chart version; make sure it's the latest before continuing.

#From Sandbox to Graduated Project

The journey toward graduated status isn't mere formality. After launching in 2019, KEDA was accepted as a CNCF sandbox project the same year, then moved up to the incubating stage, before finally being declared graduated in August 2023. This status means the project is considered mature in terms of governance, stability, and production adoption. Its contributions go far beyond Microsoft and Red Hat — AWS, Google, and dozens of other organizations participate and run KEDA in production.

The active v2.x line keeps adding scalers and stability fixes every few months. The latest release, v2.20.2, was released in July 2026, and as we'll see in episode 20, each release typically brings additional scalers plus refinements to the metrics server and operator.

#The Problems KEDA Solves

#Scaling Based on Events, Not CPU

KEDA scales based on the number of events/backlog from any source: SQS queue length, Kafka consumer lag, RabbitMQ queue depth, or the result of a Prometheus query. This makes scaling proactive — replicas increase because the cause signal appears, not because the effect (CPU) has already happened. In episodes 4 and 5 we'll see exactly how this signal is converted into a replica count.

#Scale-to-Zero: Saving 40-60%

This is probably the most sought-after feature. Because scaling decisions come from whether events exist, KEDA can drop replicas to zero when there's no work. For workloads that sit idle most of the day — batch jobs, AI inference, nightly workers — the 40-60 percent idle compute savings claim is far from exaggerated. We'll break down exactly when it's safe in episode 5.

#Native Integration with HPA

Instead of inventing a new autoscaling system, KEDA leverages existing infrastructure: it automatically creates the HPA and acts as the metrics server feeding it through the External Metrics API. No additional external dependencies, and replica scaling behavior stays controlled by the HPA mechanisms you already know.

#When HPA Alone Is Enough

As a counterbalance, not every workload needs KEDA. Use pure HPA if your target metrics really are CPU/memory or already-available custom metrics — for example, an API backend that's in constant use. Replacing HPA with KEDA for workloads like this only adds operator complexity without meaningful benefit.

Conversely, consider KEDA if at least one of these conditions applies:

Workloads are triggered by external events (queue, stream, HTTP pending requests).
You want scale-to-zero when there's no work.
The metric sources are diverse and changeable (Prometheus, SQS, Kafka, databases).
You want a single consistent autoscaling platform for all workloads.

Rule of thumb: HPA is a hammer, KEDA is a toolbox. A hammer is enough for a nail; reach for the toolbox when the jobs are varied.

#KEDA vs Other Approaches

To make KEDA's position clear, compare three common options:

Approach	Metric Source	Scale-to-zero	How It Works
Pure HPA	Resource (CPU/memory) and custom	No	Reads metrics from metrics-server
custom-metrics-adapter	Query to a specific backend (e.g. Prometheus)	No	Single adapter for one source
KEDA	70+ scalers (queue, stream, HTTP, DB)	Yes	Operator + metrics server + HPA

Pure HPA is great for classic web services. custom-metrics-adapter (e.g. the Prometheus adapter) solves the metric source problem, but doesn't solve the scale-to-zero problem and requires you to maintain the adapter yourself. KEDA combines both: one integration point for dozens of metric sources, plus the ability to scale replicas down to zero.

Tip

KEDA and HPA aren't enemies — they work together. KEDA creates and manages the HPA behind the scenes; understand HPA well and you automatically understand half of how KEDA works.

#The Right Signal for Event-Driven Workloads

There's a simple analogy to close this episode: CPU-based autoscaling is like adding cashiers after the customer line has already snaked to the door. KEDA reads the length of the queue itself — before it has a chance to snake around. For event-driven workloads, the right signal is the amount of work waiting, not how busy the workers are right now.

Comparing HPA and KEDA signals

HPA signal  : cpu over 70 percent ---> add pod (reactive)
KEDA signal : queue depth over 20 ---> add pod (proactive)

Note

Three terms that will appear often in this series: queue depth (number of messages waiting in a queue), consumer lag (production-consumption gap in streams like Kafka), and backlog (a general term for unfinished work). All three are "cause signals" that KEDA reads.

#Conclusion

Episode 1 answers the "why" question. Here's what you must bring along:

CPU-based HPA is reactive and unsuitable for event-driven workloads.
KEDA was born in 2019 from Microsoft and Red Hat; it became a graduated CNCF project in August 2023.
The latest stable version is v2.20.2 (July 2026).
KEDA scales based on events, supports scale-to-zero, and integrates natively with HPA.

In episode 2 we'll open up the engine: how the operator, metrics server, and admission webhooks work together, and what ScaledObject, ScaledJob, and TriggerAuthentication are.