U-Net Architecture
What you'll learn: How U-Net combines an encoder-decoder structure with skip connections to achieve precise pixel-level segmentation while preserving spatial details.
The Core Architecture
U-Net is named after its distinctive U-shaped architecture, designed specifically for biomedical image segmentation (though now widely used everywhere). It consists of two main paths:
The Contracting Path (Encoder):
- Standard convolutional layers with pooling
- Progressively reduces spatial dimensions
- Increases feature channels (captures what is in the image)
- Acts like a typical CNN classifier
The Expanding Path (Decoder):
- Upsampling layers (transposed convolutions)
- Progressively restores spatial dimensions
- Reduces feature channels
- Reconstructs the segmentation map
The Magic: Skip Connections
Here's what makes U-Net special: skip connections directly link encoder layers to corresponding decoder layers at the same spatial resolution.
Think of it like building a bridge: the encoder creates an abstract understanding (crossing to the other side), but the skip connections let you carry fine details directly across without losing them in translation.
Why Skip Connections Matter
When you downsample through the encoder, you lose precise spatial information—where edges and boundaries are. Skip connections concatenate high-resolution features from the encoder directly to the decoder, allowing the network to:
- Recover fine-grained spatial details
- Localize boundaries precisely
- Combine low-level features (edges, textures) with high-level semantics (what object it is)
This is crucial for segmentation tasks where you need pixel-perfect boundaries—like distinguishing between adjacent cells in microscopy or precise tumor delineation.
Key Takeaway: U-Net's encoder-decoder structure with skip connections solves segmentation's core challenge: combining semantic understanding from deep layers with spatial precision from shallow layers, making it the go-to architecture for tasks requiring detailed pixel-wise predictions.