Course contentsShow
← Machine Learning and Deep Learning
Lesson 627 of 3,53815. Neural Network FoundationsFree lesson

Forward Pass: Computing Activations Layer by Layer

Step-by-step computation of network outputs while storing intermediate activations needed for backward pass.

Forward Pass: Computing Activations Layer by Layer

What you'll learn: How neural networks systematically compute outputs by processing data through each layer sequentially, storing intermediate results needed for training.

The Core Idea

The forward pass is the sequential computation that transforms your input data into predictions. Think of it like an assembly line: raw materials (input) enter at one end, each station (layer) performs its specific transformation, and the final product (prediction) emerges at the other end. Crucially, we must keep track of what happens at each station because we'll need this information later to improve the process.

Step-by-Step Computation

When performing a forward pass, you:

  1. Start with input data X (your features)
  2. Compute first hidden layer: Apply weights, add bias, apply activation function → store result as h₁
  3. Feed h₁ to next layer: Repeat the transformation → store result as h₂
  4. Continue through all layers until you reach the output
  5. Store every intermediate activation (h₁, h₂, ..., output)

Each layer's computation follows the pattern:

z = W @ a_prev + b  # Linear transformation
a = activation(z)    # Nonlinear activation

Where a_prev is the previous layer's activation (or input for the first layer).

Why Store Activations?

You might wonder why we keep all these intermediate values. The answer lies in the computational graph: during backpropagation, we'll need these activations to calculate gradients efficiently using the chain rule. Without storing them, we'd have to recompute everything—extremely wasteful!

Key Takeaway: The forward pass computes network outputs layer by layer, storing all intermediate activations as checkpoints that enable efficient gradient computation during the backward pass.