Course contentsShow
← Computer Science
Lesson 2804 of 2,87262. Machine Learning FundamentalsFree lesson

Performance Metrics Selection

Choosing appropriate evaluation metrics (accuracy, precision, recall, F1, MSE) based on problem type.

Performance Metrics Selection

What you'll learn: How to choose the right evaluation metric to match your machine learning problem's goals and constraints.

Why the Metric Matters

Not all ways of measuring "correctness" are equal. Just as you wouldn't measure a sprinter's performance by their vertical jump height, you can't evaluate every ML model with the same metric. The right metric aligns with what actually matters in your problem domain.

Common Metrics by Problem Type

Classification Problems

  • Accuracy: The fraction of correct predictions. Use when classes are balanced and all errors cost roughly the same.
  • Precision: Of all positive predictions, how many were correct? Critical when false positives are expensive (e.g., spam filters marking important emails as spam).
  • Recall: Of all actual positives, how many did we catch? Essential when false negatives are costly (e.g., disease screening—missing a case is serious).
  • F1 Score: The harmonic mean of precision and recall. Use when you need a balanced view of both, especially with imbalanced classes.

Regression Problems

  • Mean Squared Error (MSE): Average of squared prediction errors. Penalizes large errors heavily; use when big mistakes are disproportionately bad.
  • Other variations (MAE, RMSE) exist but haven't been covered yet.

The Selection Process

Ask yourself:

  1. What's the cost structure? Are false positives and false negatives equally bad?
  2. Are classes balanced? Accuracy misleads when 95% of data is one class.
  3. What does the stakeholder care about? A doctor wants high recall; a spam system needs high precision.

Key Takeaway: Select evaluation metrics that reflect your problem's real-world costs and priorities—accuracy alone is rarely enough, especially for imbalanced or high-stakes scenarios.