Course contentsShow
← Machine Learning and Deep Learning
Lesson 2337 of 3,53850. Deep Reinforcement Learning: Advanced Policy MethodsFree lesson

World Models and Latent Imagination

Learning compressed latent representations of dynamics for efficient planning in high-dimensional spaces.

World Models and Latent Imagination

What You'll Learn

You'll discover how agents can learn compact, compressed representations of environment dynamics in a latent space, enabling efficient planning without directly processing high-dimensional observations like images.

The Core Idea

Imagine trying to plan your route through a city. You don't mentally replay every pixel of every street view—instead, you think with a compressed "mental map" that captures the essential structure. World models do the same for RL agents.

Traditional model-based RL learns dynamics in the raw observation space (e.g., predicting the next 84×84×4 Atari frame). This is computationally expensive and often contains irrelevant details. World models instead learn to:

  1. Encode high-dimensional observations into a low-dimensional latent representation (the compressed "essence" of the state)
  2. Learn dynamics in this latent space (predicting the next latent state given actions)
  3. Decode back to observations when needed
  4. Plan entirely in the efficient latent space

Latent Imagination

Latent imagination is the process of planning by "imagining" future trajectories in latent space. The agent:

  • Encodes the current observation into a latent state
  • Simulates forward using the learned latent dynamics model
  • Evaluates imagined trajectories using a value function
  • Selects actions based on these mental simulations

This is dramatically more sample-efficient than learning purely from real environment interactions, because the agent can practice thousands of imagined rollouts quickly.

Why It Matters

By compressing a 10,000-dimensional pixel observation into perhaps a 32-dimensional latent vector, planning becomes 100× faster. The latent space naturally filters out irrelevant details (like background textures) and focuses on controllable, predictive features.

Key Takeaway: World models with latent imagination compress environment dynamics into low-dimensional representations, enabling agents to plan efficiently by "imagining" trajectories in a learned latent space rather than expensive pixel space.