Course contentsShow
← Machine Learning and Deep Learning
Lesson 1468 of 3,53832. Generative Models: Autoencoders and VAEsFree lesson

VAE Training Loop in PyTorch

Implementing the complete training procedure: forward pass, ELBO loss, backward pass, and optimizer step.

VAE Training Loop in PyTorch

What You'll Learn

You'll learn how to bring together all VAE components into a complete training loop that optimizes the ELBO objective.

The Complete Training Cycle

Now that you've built the encoder, decoder, implemented the reparameterization trick, and understood ELBO, it's time to connect everything into a working training loop.

The VAE training loop follows the standard PyTorch pattern but with a custom loss calculation. Each iteration involves:

  1. Forward Pass: Feed input through encoder to get mean and log-variance, sample latent code using reparameterization, decode to reconstruct input
  2. ELBO Loss Calculation: Compute reconstruction loss (how well you rebuild the input) plus KL divergence (how much your posterior deviates from the prior)
  3. Backward Pass: Compute gradients with respect to all parameters
  4. Optimizer Step: Update encoder and decoder weights

Think of it like teaching someone to draw: they look at a picture (encoder), capture its essence in memory (latent space), then redraw it (decoder). The training loop repeatedly adjusts their "drawing skills" to both reproduce accurately and keep their mental representation organized.

optimizer.zero_grad()

# Forward pass
mu, logvar = encoder(x)
z = reparameterize(mu, logvar)
x_recon = decoder(z)

# ELBO loss
recon_loss = reconstruction_loss(x_recon, x)
kl_loss = kl_divergence(mu, logvar)
loss = recon_loss + beta * kl_loss

# Backward pass
loss.backward()

# Update weights
optimizer.step()

The key is that all components are differentiable, allowing gradients to flow from the loss back through the entire architecture.

Key Takeaway: The VAE training loop orchestrates the forward pass (encode → sample → decode), ELBO computation (reconstruction + KL), backpropagation, and weight updates in each iteration, training both encoder and decoder end-to-end.