Course contentsShow
← Machine Learning and Deep Learning
Lesson 1442 of 3,53832. Generative Models: Autoencoders and VAEsFree lesson

The Probabilistic Encoder

Learn how VAE encoders output mean and variance parameters defining a Gaussian distribution per data point.

The Probabilistic Encoder

What you'll learn: How VAE encoders differ from standard autoencoders by outputting distribution parameters instead of fixed vectors.

From Deterministic to Probabilistic

In a basic autoencoder (which you've already learned), the encoder maps an input directly to a single point in the latent space—a fixed vector. Every time you encode the same input, you get exactly the same latent representation.

Variational Autoencoders (VAEs) take a fundamentally different approach. Instead of producing one fixed point, the encoder outputs parameters that define a probability distribution. Specifically, it produces two sets of values:

  • Mean (μ): The center of a Gaussian (normal) distribution
  • Variance (σ²) or log-variance: The spread of that distribution

Think of it like describing a person's location: instead of saying "John is at coordinates (5, 3)," you say "John is somewhere around (5, 3), typically within a 2-meter radius." The mean tells you where they likely are; the variance tells you how certain you are.

Why Gaussian Distributions?

Each data point gets its own Gaussian distribution in the latent space. For a 10-dimensional latent space, the encoder outputs 10 means and 10 variances. These parameters completely specify a multivariate Gaussian distribution from which we'll later sample the actual latent vector used for reconstruction.

This probabilistic approach enables VAEs to:

  • Generate new, similar data by sampling from learned distributions
  • Capture uncertainty in the encoding
  • Create smooth, continuous latent spaces

Key Takeaway: VAE encoders are probabilistic—they output mean and variance parameters that define a Gaussian distribution for each input, rather than encoding to a single fixed point.