HPA on Custom Metrics: When CPU Tells You Nothing Useful

The HPA scales by default on CPU or memory, a relevant metric for a workload whose load translates directly into compute consumption. A worker processing a message queue doesn’t follow this pattern: it can run at 5% CPU while being completely overwhelmed by a queue that keeps growing, CPU never measuring the system’s real constraint.

The mismatch between CPU and real load

A queue consumer spends most of its time waiting (a network call, a disk write), not computing: its CPU consumption stays low even under heavy load, because load translates into queue depth, not CPU cycles spent. A CPU-based HPA for this kind of workload never scales early enough, or not at all, while the queue keeps accumulating with no limit visible to Kubernetes.

Prometheus Adapter: exposing a business metric to the HPA’s API

The HPA can only query metrics exposed through a specific Kubernetes API (custom.metrics.k8s.io), never PromQL directly. Prometheus Adapter bridges this gap: it translates a PromQL query into a response conforming to that API, letting the HPA scale on any metric already collected by Prometheus, queue depth included.

# Prometheus Adapter: translates a PromQL query
# into a metric the HPA can consume directly
rules:
  - seriesQuery: 'queue_depth{namespace!=""}'
    resources:
      overrides:
        namespace: {resource: "namespace"}
    metricsQuery: 'avg(queue_depth{<<.LabelMatchers>>})'

The HPA references the custom metric like any other

Once exposed via Prometheus Adapter, the metric gets referenced in the HPA exactly like CPU or memory, with its own target value.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: queue-worker
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: queue-worker
  minReplicas: 2
  maxReplicas: 20
  metrics:
    - type: Pods
      pods:
        metric:
          name: queue_depth
        target:
          type: AverageValue
          averageValue: "10"

This configuration scales to keep an average of 10 messages queued per replica: past that, the HPA adds replicas; below it, it removes them, the exact same calculation logic as for CPU, but applied to a metric that genuinely reflects the system’s constraint.

The trap: a poorly chosen custom metric scales on noise

An overly volatile metric (varying wildly from one second to the next with no real trend) triggers erratic scaling oscillations, the HPA reacting to statistical noise rather than a genuine load trend. A rolling average over a few minutes, rather than an instantaneous value, generally stabilizes the metric before it drives scaling that would otherwise oscillate without ever settling.

Takeaway

HPA on CPU or memory doesn’t fit every workload: a queue consumer can stay at low CPU consumption while genuinely overwhelmed, a constraint invisible to both default metrics. Prometheus Adapter bridges PromQL and the custom metrics API the HPA can consume, enabling scaling on any business metric that genuinely represents load. Choosing a stable metric (rolling average rather than instantaneous value) avoids scaling that reacts to noise rather than the real trend, one of the fine-tuning details that separate autoscaling that actually works from autoscaling that merely checks the box, central to any reliability and observability that goes beyond purely CPU-bound workloads.