Course contentsShow
← Machine Learning and Deep Learning
Lesson 769 of 3,53818. Regularization Techniques for Deep NetworksFree lesson

Mixup: Interpolating Training Examples

Blending pairs of images and labels to create synthetic examples and smooth decision boundaries.

Mixup: Interpolating Training Examples

What You'll Learn

You'll discover how Mixup creates synthetic training examples by blending pairs of images and their labels together, helping your neural network learn smoother, more robust decision boundaries.

The Core Idea

Imagine you have a picture of a cat and a picture of a dog in your training set. Instead of training on these separately, Mixup creates a new training example by literally blending them together—say, 70% cat image + 30% dog image—and creates a matching blended label: 70% "cat" + 30% "dog".

This might sound strange (what even is a 70% cat?), but it's remarkably effective! By training on these interpolated examples, your network learns to make predictions that gracefully transition between classes rather than having harsh, overconfident boundaries.

How It Works

For each training batch, Mixup:

  1. Randomly pairs examples together (image A with image B)
  2. Samples a mixing coefficient λ (lambda) from a Beta distribution, typically between 0 and 1
  3. Creates a synthetic example: new_image = λ × image_A + (1-λ) × image_B
  4. Blends the labels too: new_label = λ × label_A + (1-λ) × label_B

The Beta distribution ensures most mixtures aren't exactly 50-50 but vary naturally.

Why It Works

Mixup acts as a powerful regularizer by:

  • Forcing the network to make reasonable predictions between training examples, not just at them
  • Reducing overconfidence on training data
  • Smoothing decision boundaries between classes
  • Creating virtually infinite training variations without collecting new data

Think of it like teaching someone colors: instead of only showing pure red and pure blue, you also show purple, pink, and navy. They learn the spectrum, not just endpoints.

Key Takeaway: Mixup blends pairs of training images and labels to create synthetic examples, regularizing networks by teaching them smooth transitions between classes rather than overconfident, brittle boundaries.