Scaling Based on Custom Application Metrics

In this lesson, you will learn how to move beyond CPU and memory to scale your Kubernetes workloads based on real-world business metrics, such as incoming request rates or message queue depth.

Why Custom Metrics?

While scaling on CPU or memory is useful, it often reacts too slowly to sudden spikes in traffic. If your application processes a high-volume queue, your pods might have plenty of idle CPU, but they could be drowning in pending jobs. To solve this, we use the Prometheus Adapter. This tool acts as a bridge, translating your application-specific Prometheus metrics into a format the Kubernetes HorizontalPodAutoscaler (HPA) can understand.

The Workflow

1. Expose Metrics: Your application exposes a custom metric (e.g., http_requests_total) to Prometheus.
2. The Adapter: The Prometheus Adapter runs in your cluster. It queries your Prometheus instance and exposes these metrics through the Kubernetes custom.metrics.k8s.io API.
3. The HPA: You define an HPA resource that targets this custom metric instead of system resource utilization.

For example, if you want to scale your deployment based on the number of requests per second, your HorizontalPodAutoscaler configuration would look like this:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: queue-worker-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: queue-worker
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: 50

Making it Work

When the average requests per second across your pods exceeds 50, the HPA will automatically trigger a scale-up event. This allows your infrastructure to stay aligned with the actual "work" being performed rather than just the "power" being consumed. By utilizing the Prometheus Adapter, you transform your HPA from a reactive hardware monitor into a proactive business-logic engine that ensures your services handle load exactly when the business demands it.

Key Takeaway: By using the Prometheus Adapter to bridge custom application metrics with the Horizontal Pod Autoscaler, you can scale your services based on actual workload demands like request throughput or queue depth.