Course contentsShow
← Machine Learning and Deep Learning
Lesson 2453 of 3,53853. Speech and Audio ProcessingFree lesson

Connectionist Temporal Classification (CTC)

Learn CTC loss for sequence-to-sequence alignment without frame-level labels, enabling end-to-end ASR.

Connectionist Temporal Classification (CTC)

You'll learn how CTC enables end-to-end speech recognition without needing frame-by-frame label alignment.

The Alignment Problem

In traditional ASR, you needed to know exactly when each phoneme occurred in the audio signal—a painstaking manual annotation process. But audio frames don't neatly align with text: the word "hello" might stretch across 50 audio frames, with some frames containing silence or transition sounds.

CTC solves this: it learns to map variable-length audio sequences to variable-length text sequences without requiring frame-level timestamps.

How CTC Works

Think of CTC like a teacher grading an essay where students can write the same answer in different ways. CTC introduces a special "blank" token (often shown as -) that represents "no output yet" or "repeated character."

For the word "cat," valid CTC alignments might be:

  • c-aaa-t
  • cc-a-tt
  • ---c-at-

All collapse to "cat" after removing blanks and repeated characters.

The CTC loss sums probabilities over all possible alignments that lead to the correct transcription. The network outputs a probability distribution at each time step, and dynamic programming (the forward-backward algorithm) efficiently computes this sum.

Why It Matters for ASR

CTC enables end-to-end training: you feed raw audio features (like log-mel spectrograms) into an RNN or other sequential model, and it directly predicts text—no phoneme dictionaries, no hand-aligned training data. This dramatically simplifies the ASR pipeline.

Key Takeaway: CTC loss allows neural networks to learn sequence alignment automatically by marginalizing over all valid alignments, making truly end-to-end ASR possible without frame-level annotations.