Sequential Data and the Need for RNNs
What you'll learn: Why standard feedforward networks struggle with sequences and why temporal dependencies demand a memory mechanism.
The Problem with Feedforward Networks on Sequences
So far, you've worked with CNNs that process images—data with spatial structure but no inherent ordering. Now consider text, speech, or video: data where order matters. The sentence "dog bites man" means something entirely different from "man bites dog."
Feedforward networks (including CNNs) process inputs independently. They have no memory of previous inputs. If you feed them words one at a time, each prediction starts from scratch. The network analyzing "bites" has no idea you just showed it "dog."
Why Memory Matters: Temporal Dependencies
Sequential data contains temporal dependencies—the current element depends on what came before (and sometimes after). To predict the next word in "The sky is ___," you need context from earlier words. To classify a video action, you need information accumulated across frames.
Think of it like reading a book with amnesia: you forget the previous sentence by the time you read the next one. Comprehension becomes impossible.
The Need for Recurrent Connections
What's missing is a mechanism to carry information forward through time. You need a network that maintains an internal state—a "memory" that summarizes what it's seen so far and updates as new inputs arrive.
This is exactly what Recurrent Neural Networks (RNNs) provide: connections that loop back, allowing hidden states to persist and evolve across time steps.
Key Takeaway: Feedforward networks fail on sequences because they lack memory across time steps; RNNs introduce recurrent connections to maintain temporal context and model dependencies in sequential data like text, speech, and video.