Course contentsShow
← Machine Learning and Deep Learning
Lesson 3101 of 3,53868. Evaluation in PracticeFree lesson

Multi-Task and Multi-Objective Evaluation

Balancing and aggregating metrics across multiple objectives, handling trade-offs, and defining composite scores.

Multi-Task and Multi-Objective Evaluation

What you'll learn: How to evaluate models that optimize multiple goals simultaneously by balancing competing metrics and creating meaningful composite scores.

The Challenge of Multiple Objectives

When your ML system serves multiple purposes—like a recommendation engine that must balance relevance, diversity, and business revenue—you can't rely on a single metric. Each task or objective has its own success criteria, and they often conflict. Improving one metric might degrade another.

Think of it like designing a car: you want speed, fuel efficiency, safety, and comfort. You can't maximize all four independently—you need a strategy to balance trade-offs.

Balancing Competing Metrics

Weighted aggregation is the most common approach: assign importance weights to each metric and compute a composite score:

composite_score = w1 × metric1 + w2 × metric2 + w3 × metric3

The challenge? Choosing weights that reflect business priorities. Normalize metrics to the same scale first (0-1 range) so one doesn't dominate due to magnitude alone.

Pareto optimization is an alternative: instead of forcing a single score, identify solutions where improving one metric requires sacrificing another. You create a "frontier" of optimal trade-off points and let stakeholders choose.

Handling Trade-offs Systematically

Define primary vs secondary metrics. Your primary metric gates deployment decisions (must improve or stay neutral), while secondary metrics are monitored for degradation beyond acceptable thresholds.

For multi-task models (one model, multiple outputs), evaluate each task independently first, then assess the composite. Track per-task performance over time—a model might improve overall while silently degrading on a minority task.

Defining Composite Scores

Be transparent about aggregation logic. Document why certain metrics matter more. Use business impact as your guide: if a 1% gain in metric A is worth twice as much as metric B, reflect that in your weights.

Key Takeaway: Multi-objective evaluation requires explicit trade-off strategies—use weighted composites for actionable decisions, normalize metrics for fair comparison, and always validate that your aggregation logic aligns with real-world priorities.