Course contentsShow
← Machine Learning and Deep Learning
Lesson 920 of 3,53821. CNN ArchitecturesFree lesson

EfficientNet: Compound Scaling

Systematically scaling depth, width, and resolution together using a compound coefficient for optimal efficiency.

EfficientNet: Compound Scaling

What you'll learn: How EfficientNet balances network depth, width, and resolution together for maximum efficiency and performance.

The Core Idea

When designing CNNs, you can scale networks in three ways: make them deeper (more layers), wider (more channels per layer), or increase the input resolution (larger images). Traditionally, architects would tweak these dimensions independently, but EfficientNet introduced a smarter approach: scale all three together in a balanced way.

Think of it like tuning a car's engine. You wouldn't just increase horsepower without upgrading the transmission and tires—everything needs to work together. Similarly, if you double the input resolution, your network needs more depth and width to effectively process that additional information.

The Compound Scaling Formula

EfficientNet uses a compound coefficient φ (phi) that uniformly scales all three dimensions:

  • Depth: d = α^φ
  • Width: w = β^φ
  • Resolution: r = γ^φ

Where α, β, and γ are constants determined through a small grid search, constrained so that α · β² · γ² ≈ 2. This ensures that for each φ increment, total computational cost (FLOPs) roughly doubles.

Why It Works

Scaling one dimension hits diminishing returns quickly. A very deep but narrow network wastes potential. A high-resolution input fed to a shallow network can't extract complex features. Compound scaling ensures each dimension grows proportionally, maintaining optimal balance as you scale up model capacity.

Key Takeaway: EfficientNet's compound scaling method systematically increases depth, width, and resolution together using a single coefficient, achieving better accuracy and efficiency than scaling any dimension alone.