Swin Transformer: Hierarchical Architecture
What you'll learn: How Swin Transformer creates a pyramid of multi-scale features by progressively reducing spatial resolution through patch merging.
Why Hierarchies Matter
Remember how CNNs naturally build pyramidal feature hierarchies — early layers capture fine details at high resolution, while deeper layers capture semantic patterns at lower resolution? Vanilla ViT doesn't do this; it processes all patches at the same resolution throughout. Swin Transformer brings back this powerful hierarchical design to vision transformers.
Building the Feature Pyramid
Swin divides its architecture into stages, similar to ResNet's stage structure:
- Stage 1: Processes small patches (e.g., 4×4 pixels) at high resolution
- Patch Merging: Combines neighboring 2×2 patches into one, halving spatial dimensions
- Stage 2: Works with merged patches at half the resolution
- Repeat: Each subsequent stage merges again, creating progressively coarser features
Think of it like zooming out on a map: initially you see individual streets (fine details), then neighborhoods (mid-level patterns), then entire districts (high-level semantics).
Why This Design Wins
This hierarchical approach offers several advantages:
- Computational efficiency: Fewer tokens at deeper stages means less computation
- Multi-scale features: Different stages capture patterns at different scales
- Dense prediction tasks: Features at multiple resolutions are perfect for segmentation, detection, and other pixel-level tasks
- CNN-like flexibility: You can extract features from any stage, just like with traditional CNNs
The pyramid structure makes Swin Transformer a true backbone for computer vision, not just a classifier.
Key Takeaway: Swin Transformer's hierarchical architecture progressively reduces spatial resolution through patch merging across stages, creating a pyramid of multi-scale features that combines transformer flexibility with CNN-like efficiency and hierarchical reasoning.