Implementing Forward Propagation from Scratch
You'll learn: How to build a complete, reusable forward propagation function in NumPy that handles multiple layers with proper matrix shapes.
The Core Idea
Now that you understand the theory of forward propagation, it's time to code it yourself. A complete implementation needs to process inputs through all layers sequentially, apply activation functions correctly, handle batch dimensions properly, and return both final outputs and intermediate activations (which you'll need later for backpropagation).
Think of this as building an assembly line: raw materials (inputs) enter, pass through workstations (layers), get transformed at each step (activations), and emerge as finished products (predictions). Your code must track both the final product and what happened at each workstation.
Building the Function
Your forward propagation function should:
- Accept parameters: input data
X, a list of weight matricesW, bias vectorsb, and activation functions per layer - Initialize storage: keep a list to store activations after each layer (including the input as
a[0]) - Loop through layers: for each layer
l, computez[l] = W[l] @ a[l-1] + b[l], thena[l] = activation(z[l]) - Handle shapes carefully: ensure weight matrix dimensions match (if layer has
n_ininputs andn_outoutputs,Wshould be(n_out, n_in)) - Return outputs: both the final prediction and all intermediate activations
The key is treating layers uniformly—each follows the same pattern regardless of network depth. Batch processing happens naturally when your input X has shape (n_features, n_samples).
Key Takeaway:
A proper forward propagation implementation loops through layers systematically, applies linear transformations and activations, maintains correct matrix shapes, and stores intermediate results needed for training.