Course contentsShow
← Machine Learning and Deep Learning
Lesson 996 of 3,53822. Computer Vision TasksFree lesson

Optical Flow and Motion Estimation

Estimating pixel-wise motion between frames: using optical flow for tracking, video analysis, and motion-based segmentation.

Optical Flow and Motion Estimation

What you'll learn: How to estimate pixel-wise motion between consecutive video frames to track objects, analyze movement patterns, and segment moving regions.

What Is Optical Flow?

Optical flow is the pattern of apparent motion of objects, surfaces, and edges in a visual scene caused by relative movement between the camera and the scene. Think of it like tracking how every pixel "flows" from one frame to the next in a video—imagine placing a tiny arrow on each pixel showing where it moved.

If you've watched a time-lapse of clouds moving across the sky, optical flow would capture the direction and speed of each cloud's motion at every point in the image.

Why It Matters for Computer Vision

Once you know where each pixel moved between frames, you unlock powerful capabilities:

  • Object tracking: Follow specific objects across video frames without re-detecting them each time
  • Video analysis: Detect unusual motion patterns (like someone falling in surveillance footage)
  • Motion-based segmentation: Separate moving objects from static backgrounds by grouping pixels with similar motion vectors
  • Action recognition: Understanding human activities by analyzing motion patterns

The Core Challenge

Optical flow estimation assumes that pixel intensities remain relatively constant between frames (the brightness constancy assumption). Modern approaches use CNNs trained on synthetic datasets with ground-truth motion to learn robust motion estimation that handles real-world complications like lighting changes, occlusions, and fast motion.

The output is typically a 2D vector field—at each pixel location (x, y), you get a displacement vector (dx, dy) indicating where that pixel moved.

Key Takeaway: Optical flow provides dense, pixel-level motion information between frames, enabling sophisticated video understanding tasks like tracking, motion segmentation, and activity analysis by treating motion itself as a rich signal.