Course contentsShow
← Machine Learning and Deep Learning
Lesson 984 of 3,53822. Computer Vision TasksFree lesson

Semantic Segmentation Datasets

Explore standard datasets like Cityscapes, PASCAL VOC, and ADE20K with their annotation formats and challenges.

Semantic Segmentation Datasets

What you'll learn: You'll explore the standard datasets used to train and evaluate semantic segmentation models, understanding their unique characteristics and challenges.

Why Datasets Matter for Segmentation

Just as you need labeled examples to learn any task, semantic segmentation models require carefully annotated images where every pixel has a class label. Unlike classification (one label per image) or detection (bounding boxes), segmentation demands pixel-perfect annotations—a painstaking, expensive process.

The Major Players

Cityscapes

Cityscapes focuses on urban street scenes from a car's perspective. It contains high-resolution images (1024×2048) with 30+ classes like road, sidewalk, car, pedestrian, and traffic signs. The challenge? Fine-grained detail—distinguishing a pole from a traffic light or handling partially occluded objects. Think of it as teaching a self-driving car to "see" like a human driver.

PASCAL VOC

An older but foundational dataset, PASCAL VOC (Visual Object Classes) includes 20 object categories plus background. Images are lower resolution but diverse—indoor, outdoor, various objects. The annotation format is simpler, making it excellent for learning. However, its smaller size (around 2,000 training images) means models can overfit without augmentation.

ADE20K

This is the most diverse dataset with 150 classes covering indoor and outdoor scenes. Images range from bedrooms to city skylines, challenging models to handle extreme variety. The trade-off? More classes mean more label ambiguity and harder learning.

Common Challenges

All segmentation datasets face similar issues: class imbalance (sky pixels vastly outnumber bicycle pixels), boundary precision (edges between objects), and annotation consistency across different human annotators.

Key Takeaway: Cityscapes excels for autonomous driving, PASCAL VOC for foundational learning, and ADE20K for handling diverse real-world scenes—each dataset shapes what segmentation models learn to see and where they excel.