Unrolling RNNs Through Time
What you'll learn: How to visualize an RNN as a deep network by "unrolling" its recurrent connections across multiple time steps.
The Core Idea
Remember that a basic RNN cell processes one input at a time and maintains a hidden state. When we talk about unrolling through time, we're taking that single recurrent cell and drawing it out across all the time steps in our sequence—like unrolling a scroll to see the full story.
Imagine you have a sentence: "The cat sat." That's three words (three time steps). Instead of thinking about one RNN cell that reuses itself three times, we unfold it into what looks like three separate cells connected in sequence—one for "The," one for "cat," and one for "sat."
Why This Visualization Matters
When unrolled, an RNN suddenly looks like a deep feedforward network, where:
- Each "layer" corresponds to one time step
- Information flows left-to-right (forward in time) through hidden states
- The same weights are shared across all time steps (this is crucial!)
This unrolled view helps us understand:
- How information from early time steps influences later predictions
- Why gradients can vanish or explode over long sequences (you're essentially backpropagating through a very deep network)
- How we actually train RNNs using backpropagation through time (BPTT)
The Transformation
One recurrent cell → Multiple copies across time → A deep sequential network
Each unrolled copy processes its input and passes its hidden state to the next time step, creating a chain of computations.
Key Takeaway: Unrolling an RNN transforms the conceptual loop into a visual deep network across time, making it easier to understand both forward computation and gradient flow during training.