Forward Pass Through Multi-Layer Networks
What You'll Learn
You'll learn how neural networks make predictions by flowing input data through multiple layers of transformations, combining linear operations with nonlinear activations at each step.
The Core Idea
A forward pass through a multi-layer network is like a factory assembly line where raw materials (input data) pass through multiple workstations (layers), each performing a specific transformation, until you get the final product (prediction).
At each layer:
- Affine transformation: Multiply inputs by weights and add biases (
z = Wx + b) - Nonlinear activation: Apply a function like ReLU or sigmoid (
a = σ(z)) - Pass to next layer: The output becomes the input for the next layer
For a network with layers L₁, L₂, ..., Lₙ:
Input x → [Linear + Activation]₁ → [Linear + Activation]₂ → ... → [Linear + Activation]ₙ → Output ŷ
Each layer's output is called an activation or intermediate activation. These activations store increasingly abstract representations of your input as information flows deeper into the network.
Why Multiple Layers?
Each successive layer builds on the previous one's work. Early layers might detect simple patterns (edges in images, common words in text), while deeper layers combine these into complex concepts (faces, sentiment). This hierarchical composition is what makes deep networks powerful.
The forward pass is entirely deterministic—given the same input and weights, you always get the same output. It's the prediction phase where all your learned parameters come together to transform inputs into useful predictions.
Key Takeaway: Forward propagation computes predictions by repeatedly applying affine transformations (Wx + b) followed by nonlinearities at each layer, passing intermediate activations forward until reaching the output layer.