Course contentsShow
← AI Engineering
Lesson 1666 of 1,88640. Computer Vision DeploymentFree lesson

Temporal Smoothing and Tracking

Apply temporal filtering and object tracking to reduce prediction jitter and maintain consistent identities across frames.

Temporal Smoothing and Tracking

What you'll learn: How to eliminate jittery predictions and maintain stable object identities across video frames using temporal filtering and tracking algorithms.

The Problem: Prediction Jitter

When running frame-by-frame inference on video, you've likely noticed predictions "jumping around" — bounding boxes flicker, classifications oscillate between classes, and detected objects appear and disappear randomly. This happens because each frame is processed independently without considering temporal context.

Temporal Smoothing

Temporal smoothing applies filtering across consecutive frames to stabilize predictions. Think of it like noise-canceling headphones for your model's output — instead of believing every single prediction, you average or weight recent history.

Common approaches:

  • Moving average: Average the last N predictions (positions, class scores)
  • Exponential smoothing: Weight recent frames more heavily than distant ones
  • Kalman filtering: Use motion models to predict where objects should be, correcting for measurement noise

For example, if a bounding box jumps 5 pixels left, 3 right, then 4 left across three frames, smoothing reveals the object is actually moving slowly leftward rather than spastically bouncing.

Object Tracking

Tracking maintains consistent identities for objects across frames. Instead of treating frame 10's "person" as unrelated to frame 9's "person," tracking algorithms link them together.

Key techniques:

  • IoU matching: Link detections with high overlap across frames
  • Feature-based tracking: Match objects using appearance embeddings
  • Motion prediction: Anticipate where objects will move next

Tracking prevents ID switches (person #3 becoming person #7) and handles temporary occlusions by predicting positions even when detection fails.

The Combination Effect

Together, smoothing and tracking create stable, production-grade video inference: bounding boxes glide smoothly, classifications remain consistent, and objects maintain their identity throughout the video — even if your per-frame detector occasionally hiccups.

Key Takeaway: Temporal smoothing filters noisy predictions across frames while tracking maintains object identities, transforming jittery frame-by-frame inference into stable, coherent video understanding.