GhostNet: Cheap Operations for Redundant Features
What you'll learn: How GhostNet reduces computational cost by generating redundant feature maps through cheap linear operations instead of expensive convolutions.
The Core Insight: Feature Map Redundancy
When you look at feature maps produced by CNNs, researchers discovered that many channels contain similar or redundant information. GhostNet exploits this by asking: "Why compute similar features multiple times with expensive convolutions when we can generate them cheaply from existing features?"
How GhostNet Works
Instead of producing all feature maps directly through standard convolutions, GhostNet:
- Creates a subset of intrinsic features using regular convolutions (fewer channels)
- Generates "ghost" features by applying cheap linear operations (like depthwise convolutions) to those intrinsic features
- Concatenates the intrinsic and ghost features to create the final output
Think of it like photocopying: instead of drawing 10 similar sketches by hand (expensive), you draw 2-3 originals carefully, then make copies with slight variations (cheap).
The Analogy
Imagine you're a chef creating dishes. Traditional CNNs cook every dish from scratch (expensive). GhostNet cooks a few base dishes thoroughly, then creates variations by adding different simple garnishes or sauces (cheap operations). The result? A full menu at a fraction of the original cost.
The Mathematics
If you need m output channels traditionally requiring m expensive convolutions, GhostNet produces only m/s intrinsic features, then generates (s-1) ghost features per intrinsic feature using cheap operations, achieving similar output with significantly fewer parameters and FLOPs.
Key Takeaway: GhostNet dramatically reduces computational cost by generating redundant feature maps through cheap linear transformations rather than expensive convolutions, maintaining accuracy while improving efficiency—ideal for mobile and edge deployment.