The Probabilistic Encoder
What you'll learn: How VAE encoders differ from standard autoencoders by outputting distribution parameters instead of fixed vectors.
From Deterministic to Probabilistic
In a basic autoencoder (which you've already learned), the encoder maps an input directly to a single point in the latent space—a fixed vector. Every time you encode the same input, you get exactly the same latent representation.
Variational Autoencoders (VAEs) take a fundamentally different approach. Instead of producing one fixed point, the encoder outputs parameters that define a probability distribution. Specifically, it produces two sets of values:
- Mean (μ): The center of a Gaussian (normal) distribution
- Variance (σ²) or log-variance: The spread of that distribution
Think of it like describing a person's location: instead of saying "John is at coordinates (5, 3)," you say "John is somewhere around (5, 3), typically within a 2-meter radius." The mean tells you where they likely are; the variance tells you how certain you are.
Why Gaussian Distributions?
Each data point gets its own Gaussian distribution in the latent space. For a 10-dimensional latent space, the encoder outputs 10 means and 10 variances. These parameters completely specify a multivariate Gaussian distribution from which we'll later sample the actual latent vector used for reconstruction.
This probabilistic approach enables VAEs to:
- Generate new, similar data by sampling from learned distributions
- Capture uncertainty in the encoding
- Create smooth, continuous latent spaces
Key Takeaway: VAE encoders are probabilistic—they output mean and variance parameters that define a Gaussian distribution for each input, rather than encoding to a single fixed point.