Course contentsShow
← Machine Learning and Deep Learning
Lesson 3066 of 3,53867. A/B Testing and Deployment StrategiesFree lesson

Proxy Metrics and North Star Metrics

Identify proxy metrics that can be measured quickly while correlating with long-term business goals.

Proxy Metrics and North Star Metrics

What you'll learn: How to identify measurable proxy metrics that correlate with long-term business goals when immediate feedback isn't available.

The Challenge of Delayed Feedback

In production ML systems, your ultimate business goal (the "North Star Metric") might take weeks or months to measure. For example, a recommendation system's true success is measured by customer lifetime value, but you can't wait 6 months to know if today's model change worked. You need proxy metrics — faster signals that correlate with your North Star.

What Are Proxy Metrics?

Proxy metrics are measurable indicators you can track quickly (minutes to days) that historically correlate with your long-term business objectives. Think of them as leading indicators: a thermometer reading (proxy) tells you about a fever before blood test results (North Star) come back.

For a content recommendation system:

  • North Star Metric: 90-day user retention rate
  • Proxy Metrics: click-through rate, session duration, content completion rate, daily active users

The key is establishing and validating the correlation between your proxies and North Star through historical analysis.

Selecting Good Proxy Metrics

  1. Measurable quickly: Available within hours or days, not months
  2. Correlated: Strong statistical relationship with the North Star (validated on historical data)
  3. Sensitive: Changes when model quality changes
  4. Actionable: Points toward specific improvements when degraded

For example, if immediate click-through rate drops correlate with eventual churn, it becomes a valuable proxy worth monitoring in real-time dashboards.

Validation Is Critical

Never assume a proxy works — validate it. Analyze historical periods where the proxy moved and verify the North Star followed. If correlation breaks down, find better proxies or wait for ground truth.

Key Takeaway: Proxy metrics bridge the gap between real-time monitoring and long-term business impact by providing fast, correlated signals that let you detect and respond to problems before they affect your North Star metric.