What Is Semantic Segmentation
What you'll learn: How semantic segmentation differs from classification and object detection by assigning a class label to every single pixel in an image.
The Core Idea
Imagine you're not just identifying what's in an image (classification) or where objects are with boxes (object detection), but rather coloring every pixel according to what it represents. That's semantic segmentation.
Semantic segmentation is a pixel-wise classification task where the goal is to assign a class label to each pixel in an image. Instead of saying "there's a cat at coordinates (x, y, w, h)" or "this is an image of a street scene," you're labeling whether pixel (10, 25) is "road," pixel (11, 25) is "car," pixel (100, 200) is "person," and so on.
How It Differs From What You Know
- Classification: Assigns one label to the entire image ("cat" or "dog")
- Object Detection: Draws bounding boxes around objects and labels them ("cat at [50, 30, 100, 80]")
- Semantic Segmentation: Labels every pixel with its class ("this pixel is sky, that pixel is tree, this pixel is road")
Real-World Analogy
Think of classification as naming a photograph, object detection as outlining objects with rectangles, and semantic segmentation as a detailed paint-by-numbers where you color each tiny section with the correct category color.
Important Note
Semantic segmentation identifies categories but doesn't distinguish between individual instances. All cars get the same "car" label, even if there are multiple separate vehicles.
Key Takeaway: Semantic segmentation performs pixel-wise classification, producing dense predictions where every pixel receives a class label, making it ideal for tasks requiring precise spatial understanding like autonomous driving and medical imaging.