Connectionist Temporal Classification (CTC)
You'll learn how CTC enables end-to-end speech recognition without needing frame-by-frame label alignment.
The Alignment Problem
In traditional ASR, you needed to know exactly when each phoneme occurred in the audio signal—a painstaking manual annotation process. But audio frames don't neatly align with text: the word "hello" might stretch across 50 audio frames, with some frames containing silence or transition sounds.
CTC solves this: it learns to map variable-length audio sequences to variable-length text sequences without requiring frame-level timestamps.
How CTC Works
Think of CTC like a teacher grading an essay where students can write the same answer in different ways. CTC introduces a special "blank" token (often shown as -) that represents "no output yet" or "repeated character."
For the word "cat," valid CTC alignments might be:
c-aaa-tcc-a-tt---c-at-
All collapse to "cat" after removing blanks and repeated characters.
The CTC loss sums probabilities over all possible alignments that lead to the correct transcription. The network outputs a probability distribution at each time step, and dynamic programming (the forward-backward algorithm) efficiently computes this sum.
Why It Matters for ASR
CTC enables end-to-end training: you feed raw audio features (like log-mel spectrograms) into an RNN or other sequential model, and it directly predicts text—no phoneme dictionaries, no hand-aligned training data. This dramatically simplifies the ASR pipeline.
Key Takeaway: CTC loss allows neural networks to learn sequence alignment automatically by marginalizing over all valid alignments, making truly end-to-end ASR possible without frame-level annotations.