Course contentsShow
← Machine Learning and Deep Learning
Lesson 2465 of 3,53853. Speech and Audio ProcessingFree lesson

Tacotron Architecture

Learn Tacotron's encoder-attention-decoder structure for generating mel spectrograms from text.

Tacotron Architecture

What you'll learn: How Tacotron uses an encoder-attention-decoder structure to transform text into mel spectrograms for speech synthesis.

The Core Innovation

Tacotron is an end-to-end neural text-to-speech model that directly converts text into mel spectrograms—the intermediate acoustic representation we covered earlier. Unlike traditional pipelines requiring multiple hand-crafted components, Tacotron learns the entire mapping in one unified architecture.

The Three-Part Structure

Think of Tacotron like a translation system that converts written language into visual speech patterns:

1. Encoder

The encoder processes input text character-by-character (or phoneme-by-phoneme). It uses:

  • An embedding layer to convert characters into vectors
  • Convolutional layers to capture local patterns
  • A bidirectional RNN (often LSTM or GRU) to build context-aware representations of the text

2. Attention Mechanism

The attention layer acts as a "soft alignment" between input text and output frames. As the decoder generates each mel spectrogram frame, attention decides which part of the input text to focus on—learning naturally where to "read from" at each timestep.

3. Decoder

An autoregressive RNN decoder generates mel spectrogram frames one at a time. Each frame prediction:

  • Conditions on the attention context (what text to speak now)
  • Uses previously generated frames (creating natural prosody)
  • Outputs multiple mel frequency bins per timestep

The decoder also predicts a "stop token" to signal when synthesis should end.

Why It Works

Tacotron's encoder-attention-decoder design learns pronunciation, timing, and prosody jointly from <text, audio> pairs, eliminating the need for phonetic dictionaries or duration models that plagued earlier systems.

Key Takeaway: Tacotron's encoder-attention-decoder architecture transforms text into mel spectrograms through learned alignment, enabling end-to-end neural speech synthesis without manual linguistic feature engineering.