Course contentsShow
← Machine Learning and Deep Learning
Lesson 985 of 3,53822. Computer Vision TasksFree lesson

Multi-Scale Inference and Test-Time Augmentation

Learn techniques for combining predictions at multiple scales and orientations to improve segmentation accuracy.

Multi-Scale Inference and Test-Time Augmentation

What You'll Learn

You'll learn how to boost segmentation accuracy by combining predictions from multiple scales and augmented versions of your test images.

The Core Idea

When you train a semantic segmentation model, it learns to recognize patterns at a particular scale. But real-world images contain objects at varying sizes and orientations. Multi-scale inference means running your trained model on the same image at different resolutions (scales), then combining the results. Test-time augmentation (TTA) extends this by also flipping, rotating, or adjusting the image, predicting on each variation, and averaging the predictions.

Think of it like getting multiple expert opinions: one doctor examines you standing up, another while sitting, and a third checks your reflection in a mirror. Each perspective reveals slightly different details, and combining their diagnoses gives you more confidence.

How It Works

For multi-scale inference:

  1. Resize your test image to multiple scales (e.g., 0.5×, 1.0×, 1.5×)
  2. Run segmentation on each scaled version
  3. Resize all predictions back to the original size
  4. Average or vote across the predictions

For test-time augmentation:

  1. Create augmented versions (horizontal flip, rotations, brightness adjustments)
  2. Get predictions for each version
  3. Reverse the augmentation on predictions (un-flip, un-rotate)
  4. Combine all predictions through averaging or voting

This leverages the same multi-scale reasoning you learned about in Multi-Scale Feature Detection and SSD: Multi-Scale Feature Maps, but applied at inference time rather than being built into the architecture.

The trade-off? Significantly longer inference time (proportional to the number of scales/augmentations), but often 1-3% better accuracy on segmentation benchmarks.

Key Takeaway: Multi-scale inference and test-time augmentation improve segmentation by combining predictions from different views of the same image, trading computational cost for increased robustness and accuracy.