Course contentsShow
← Machine Learning and Deep Learning
Lesson 1008 of 3,53823. Recurrent Neural NetworksFree lesson

One-to-Many RNN Architecture

Generating sequences from fixed inputs: image captioning initialization and conditional generation.

One-to-Many RNN Architecture

What you'll learn: How RNNs can generate sequences of outputs from a single fixed input, enabling tasks like image captioning.

The Core Idea

In a One-to-Many RNN architecture, you start with a single fixed input (like an image) and generate a sequence of outputs (like words describing that image). This is the opposite of Many-to-One, where you compressed a sequence into one output.

Think of it like a storyteller who looks at a photograph once, then describes it word by word: "A," "cat," "sitting," "on," "a," "chair." The photo is your single input, and each word flows from the RNN's hidden state as it unfolds through time.

How It Works

The architecture begins by initializing the RNN's hidden state with information from your fixed input. For image captioning, a CNN processes the image into a feature vector, which either:

  1. Directly initializes the RNN's first hidden state (h₀), or
  2. Is fed as context at each time step alongside generated tokens

Once initialized, the RNN generates outputs autoregressively: each predicted word becomes the input for the next time step. The hidden state carries forward, maintaining context about what's been generated so far.

This is conditional generation—you're not generating random sequences, but sequences conditioned on your initial input (the image features).

Image Captioning Example

  1. CNN extracts image features → vector representation
  2. This vector initializes the RNN's hidden state
  3. RNN generates <start> → "A" → "dog" → "playing" → <end>
  4. Each word is fed back as input for the next step

Key Takeaway: One-to-Many RNNs generate variable-length sequences from fixed inputs by conditioning the sequence generation on an initial context, making them ideal for image captioning and similar tasks where you translate from one domain to another.