Course contentsShow
← Machine Learning and Deep Learning
Lesson 3128 of 3,53868. Evaluation in PracticeFree lesson

Why Aggregate Metrics Hide Problems

Learning how high overall accuracy can mask poor performance on critical subpopulations or edge cases.

Why Aggregate Metrics Hide Problems

What you'll learn: How high overall accuracy can mask poor performance on critical subpopulations or edge cases.

The Illusion of Success

Imagine a medical AI with 95% accuracy detecting diseases. Sounds impressive, right? But what if 90% of your data comes from one demographic, and the model only achieves 60% accuracy on underrepresented groups? The high aggregate metric hides a critical failure.

Aggregate metrics like overall accuracy, average precision, or mean squared error compute a single number across your entire test set. They're convenient for comparing models quickly, but they average away important patterns of failure.

Why This Matters

When you collapse performance into one number, you lose visibility into:

  • Subpopulation disparities: A fraud detector might excel on common transaction types but fail on rare, high-value cases
  • Edge case blindness: Self-driving car models might perform well overall but catastrophically fail in rain or fog
  • Class imbalance effects: 99% accuracy means nothing if your model just predicts "negative" for everything in a 99:1 imbalanced dataset
  • Fairness issues: Different demographic groups may experience vastly different model quality

The Hidden Reality

A model with 92% accuracy might be:

  • 98% accurate on the majority class (safe to deploy)
  • 45% accurate on a critical minority slice (dangerous failure)

The aggregate obscures this gap. You might ship a model that works beautifully for most users while systematically failing for others—often the most vulnerable populations or highest-stakes scenarios.

Key Takeaway: Aggregate metrics provide false confidence; always decompose performance across slices, subgroups, and edge cases to reveal where your model truly struggles before deployment.