Autoscaling Policies based on Custom Metrics

In this lesson, you will learn how to transcend basic CPU-based autoscaling by triggering Horizontal Pod Autoscalers (HPA) using custom application metrics like .NET request queue depth.

Beyond CPU: Why Custom Metrics Matter

Standard HPA configurations often rely on Resource metrics like CPU or memory. While useful, these are "lagging" indicators—by the time your CPU spikes, your request queue may already be saturated, leading to 503 errors. For high-performance .NET microservices, scaling based on the actual load—such as the number of pending requests in an IQueue or an internal buffer—provides a much more reactive and accurate infrastructure footprint.

The Mechanism

To scale based on custom metrics, you need a bridge between your application and the Kubernetes metrics pipeline. The most common pattern is using the Prometheus Adapter. Your .NET application exposes these metrics via the prometheus-net library:

// Expose a custom gauge for request queue length
var queueGauge = Metrics.CreateGauge("myapp_request_queue_depth", "Current depth of request queue");

public void ProcessRequests() {
    queueGauge.Set(myInternalQueue.Count);
    // ...
}

Once Prometheus scrapes this value, the Custom Metrics API makes it available to Kubernetes. You then define an HorizontalPodAutoscaler that points to this specific metric rather than just cpu:

kind: HorizontalPodAutoscaler
spec:
  scaleTargetRef:
    kind: Deployment
    name: net-microservice
  metrics:
  - type: Pods
    pods:
      metric:
        name: myapp_request_queue_depth
      target:
        type: AverageValue
        averageValue: 50

Strategic Tuning

When setting your averageValue (e.g., 50), consider your application’s throughput profiling. If your service processes requests efficiently up to 50 items in the queue before latency degrades, setting that as the target threshold allows Kubernetes to spin up new pods just as the "pressure" begins to build, rather than waiting for the underlying hardware to report high utilization. This proactive approach ensures your .NET runtime maintains a consistent latency profile even under unpredictable load.

Key Takeaway: Moving from resource-based scaling to custom application metrics allows you to scale your .NET pods based on business-specific pressure indicators, ensuring high availability before infrastructure constraints manifest as performance bottlenecks.