Introducing Kubernetes Event-driven Autoscaling (KEDA)

Kubernetes Native Horizontal Pod Autoscaling Support

Kubernetes natively supports Horizontal Pod Autoscaling: it enables engineers to define automatic scaling conditions and behaviour for target workloads by using the HorizontalPodAutoscaler object from the Kubernetes autoscaling API Group. Kubernetes uses these user-defined configurations to dynamically adjust the number of pod replicas for specified workloads based on real-time demand.

Our Horizontal Pod Autoscaling series explains how to get the most out of Kubernetes HPA.

Despite this built-in functionality, the Cloud Native Computing Foundation (CNCF) accepted the Kubernetes Event-Driven Autoscaling (KEDA) as a CNCF project in March 2020. In August 2023 KEDA became a Graduated CNCF project, marking the tool as a production-ready solution for autoscaling Kubernetes workloads.

What is Kubernetes Event-driven Autoscaling (KEDA)?

Microsoft announced they had partnered with Red Hat to create KEDA in May 2019, as a tool built on top of the Kubernetes Horizontal Pod Autoscaler (HPA) without replacing it. KEDA is an event-driven autoscaler, focused on enabling Kubernetes applications to scale automatically in response to events that have occurred in an event source (a KEDA Scaler).

For example, you can have a MySQL database as an external event source. By granting KEDA access to the database with a valid query, KEDA can watch the current value returned by the query and automatically scale a target Kubernetes workload up if the returned value exceeds a user-defined threshold and scale then scale it back down if the value falls below the threshold.

HPA vs KEDA

KEDA uses the Kubernetes HPA to help deliver some of its event driven autoscaling capabilities, but also offers additional functionality to fill in the gaps of the Kubernetes HPA. In this section, I will be explaining some of differences between HPA and KEDA.

Metric APIService Restrictions

Metrics Server, the common cluster add-on installed on many Kubernetes clusters, collects resource metrics exposed by the kubelet process running in each node. It’s simple, fast, efficient and scalable. Kubernetes HPA can automatically scale workloads based on metrics collected by the Metrics Server such as CPU and memory usage metrics. However, the Metrics Server only serves the metrics-k8s.io API group.

Using only Kubernetes HPA to automatically scale workloads using custom metrics (custom.metrics.k8s.io - metrics related to objects in the Kubernetes cluster) or external metrics (external.metrics.k8s.io - metrics external to a Kubernetes cluster) coming from a single data source can also be considered viable, but you will need to install metrics adapters - components that will connect your data source (e.g. a monitoring system) to Kubernetes to provide access to the custom or external metrics from the data source.

However, each of these metric APIs (metrics-k8s.io, custom.metrics.k8s.io, external.metrics.k8s.io) registered onto a Kubernetes cluster as a APIService Kubernetes object to represent different types/sources of metric data:

• Installing the Metrics Server will involve the creation of an APIService called v1beta1.metrics.k8s.io
• Installing a metrics adapter will involve the creation of an APIService called v1beta1.custom.metrics.k8s.io or v1beta1.external.metrics.k8s.io.

Some adapters will create both.

$ kubectl get apiservices | grep metrics
NAME                              SERVICE
v1beta1.custom.metrics.k8s.io     default/prom-adapter-prometheus-adapter
v1beta1.external.metrics.k8s.io   default/datadog-monitoring-cluster-agent-metrics-api
v1beta1.metrics.k8s.io            default/metrics-server

⚠️ At the time of writing May 2025, there can only be one APIService (data source) tied to each metric API, but there is an issue opened in 2020 requesting for support for multiple custom/external metrics servers in a cluster.

As a result, the number of data sources you can use to fetch custom and external metrics from is limited. You can choose to funnel all your metrics into one location (like Prometheus) and install a Prometheus metrics adapter in your Kubernetes cluster.

In contrast, KEDA has a more flexible yet simpler approach: you just need to install KEDA and then configure KEDA to use metrics and/or events from any event source supported by KEDA.

CNCF has a blog post from members of the Alibaba Cloud EDAS team about why Alibaba Cloud chose to use KEDA for application autoscaling which contains diagrams illustrating this problem.

KEDA extends HPA

KEDA does not replace the Kubernetes HPA, instead it uses HPA under the hood to provide certain features and extend its functionality without overwriting or duplicating the Kubernetes native autoscaling capabilities HPA provides. If you find yourself using KEDA you may find HorizontalPodAutoscaler objects created by KEDA, but not all KEDA features require an HPA object.

KEDA supports a wide range of event sources, known as built-in KEDA scalers, including the Kubernetes Metrics Server, meaning you can also configure KEDA to automatically scale workloads using CPU and Memory provided by the Kubernetes Metrics Server. But, instead of defining a HorizontalPodAutoscaler object, you will need to deploy a KEDA custom resource.

Scaling to or from Zero

One of the features KEDA provides is its ability to scale workloads to or from zero, which is especially useful when no events are occurring that require processing and you want to optimise resource usage:

Since Kubernetes v1.16, there has been a feature gate called HPAScaleToZero to enable setting the minimum number of replicas to 0 when using custom or external metrics, but it is disabled by default at the time of writing (Kubernetes v1.32).

Scaling Custom Resources

With KEDA you can also automatically workloads defined as a Kubernetes custom resources, but the Custom Resource Definition (CRD) must define the /scale subresource.

Metric/Event Sources

At the time of writing, KEDA has built-in support for fetching metrics and events from 74 event sources (which include Azure Monitor, Kafka, RabbitMQ, AWS SQS, and more) and with KEDA there is a standardised approach for configuring autoscaling for any of the supported event sources using two Kubernetes custom resources: ScaledObject and ScaledJob.

Assuming you already have KEDA installed, the process for setting up autoscaling for a given KEDA scaler (event source) is straightforward, you only have to deploy a correctly configured ScaledObject or ScaledJob. To get the configurations correct for the given scaler you can lean on KEDA’s documentation where each built-in scaler has it’s own page outlining all the parameters you need to define in your ScaledObject or ScaledJob object.

If your desired event source is not supported, KEDA does support external scalers, but you will have to build and maintain a GRPC service that implements the same interface as a built-in scaler.

Summary: When to use KEDA

If you only need your application scaling based on basic metrics like CPU and memory or if you only need to use custom or external metrics from a single data source, then it’s OK to use Kubernetes’s native Horizontal Pod Autoscaler.

You should consider KEDA if:

• You are looking to cut down on costs by making use of KEDA’s feature to scale-to-zero when dealing with idle workloads or empty queues
• Significant engineering effort will need to be invested into building, installing and maintaining the multiple services needed to consolidate data into a single source that can be exposed to your Kubernetes clusters through a limited set of metrics adapters. This is especially an issue if:
  • You’re working with external systems that don’t expose metrics easily
  • Your preferred monitoring solution doesn’t have a maintained HPA metrics adapter (for example, the AWS CloudWatch adapter is deprecated).
• You want to automatically scale a custom resource
• You want more options for fine-tuning how your workloads are automatically being scaled.

If have decided to explore using KEDA, in the next blog post, we will explain how to get started with KEDA and the key concepts you need to know before you start using KEDA.