Course contentsShow
← Machine Learning and Deep Learning
Lesson 1462 of 3,53832. Generative Models: Autoencoders and VAEsFree lesson

Decoder Architecture and Output Activation

Designing decoders to reconstruct inputs, choosing appropriate final activations (sigmoid, tanh, or none) per data type.

Decoder Architecture and Output Activation

What you'll learn: How to design VAE decoders that successfully reconstruct inputs by choosing the right architecture and final activation function.

The Decoder's Job

The decoder in a VAE is a generative model that takes a latent vector and reconstructs the original input. Think of it as a mirror image of your encoder: if the encoder compressed 784 pixels down to a 20-dimensional latent space, your decoder must expand those 20 numbers back to 784 pixels.

Architectural Design

Your decoder architecture typically mirrors the encoder in reverse:

  • Encoder: Linear layers that progressively shrink dimensions (e.g., 784 → 400 → 200 → 20)
  • Decoder: Linear layers that progressively expand dimensions (e.g., 20 → 200 → 400 → 784)

For convolutional VAEs, use transposed convolutions or upsampling layers to reverse the spatial downsampling your encoder performed.

Choosing the Output Activation

This is critical and depends entirely on your data type:

  • Sigmoid: Use when reconstructing data in [0, 1] range (e.g., normalized images, binary data). Squashes outputs between 0 and 1.
  • Tanh: Use when your data is in [-1, 1] range. Squashes outputs between -1 and 1.
  • None (linear): Use when reconstructing unbounded continuous data (e.g., real-valued features). Lets the network output any value.

Mismatching your activation to your data range will prevent proper reconstruction—using sigmoid on [-1, 1] data means the network can never output negative values!

Key Takeaway: Design your VAE decoder as a mirror of the encoder, and match your final activation function to your data's range—sigmoid for [0,1], tanh for [-1,1], or none for unbounded data.