One-to-Many RNN Architecture
What you'll learn: How RNNs can generate sequences of outputs from a single fixed input, enabling tasks like image captioning.
The Core Idea
In a One-to-Many RNN architecture, you start with a single fixed input (like an image) and generate a sequence of outputs (like words describing that image). This is the opposite of Many-to-One, where you compressed a sequence into one output.
Think of it like a storyteller who looks at a photograph once, then describes it word by word: "A," "cat," "sitting," "on," "a," "chair." The photo is your single input, and each word flows from the RNN's hidden state as it unfolds through time.
How It Works
The architecture begins by initializing the RNN's hidden state with information from your fixed input. For image captioning, a CNN processes the image into a feature vector, which either:
- Directly initializes the RNN's first hidden state (h₀), or
- Is fed as context at each time step alongside generated tokens
Once initialized, the RNN generates outputs autoregressively: each predicted word becomes the input for the next time step. The hidden state carries forward, maintaining context about what's been generated so far.
This is conditional generation—you're not generating random sequences, but sequences conditioned on your initial input (the image features).
Image Captioning Example
- CNN extracts image features → vector representation
- This vector initializes the RNN's hidden state
- RNN generates
<start>→ "A" → "dog" → "playing" →<end> - Each word is fed back as input for the next step
Key Takeaway: One-to-Many RNNs generate variable-length sequences from fixed inputs by conditioning the sequence generation on an initial context, making them ideal for image captioning and similar tasks where you translate from one domain to another.