Course contentsShow
← Machine Learning and Deep Learning
Lesson 50 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

The Jacobian Matrix

Organizing partial derivatives of vector-valued functions, essential for neural network layers.

The Jacobian Matrix

What You'll Learn

You'll discover how to organize all partial derivatives when a function takes in multiple inputs and produces multiple outputs—crucial for understanding how neural network layers transform data.

The Core Idea

Imagine you have a function that takes a vector as input and produces a vector as output. For example, a neural network layer might take 3 inputs and produce 2 outputs. The Jacobian matrix collects all the partial derivatives that describe how each output depends on each input.

Think of it as a "sensitivity map": each entry tells you how much one specific output changes when you nudge one specific input, holding everything else constant.

Structure of the Jacobian

If you have a function f that maps from ℝⁿ to ℝᵐ (n inputs, m outputs), the Jacobian is an m × n matrix:

  • Rows correspond to outputs
  • Columns correspond to inputs
  • Entry at row i, column j = ∂fᵢ/∂xⱼ (how output i responds to input j)

Example: A Simple Transformation

Suppose f(x, y) = [x² + y, xy]. This takes 2 inputs and gives 2 outputs.

The Jacobian is:

J = [ ∂f₁/∂x   ∂f₁/∂y ]   =  [ 2x   1 ]
    [ ∂f₂/∂x   ∂f₂/∂y ]      [ y    x ]

Each row shows the gradient of one output function with respect to all inputs.

Why It Matters for Neural Networks

In deep learning, you chain many layers together. The Jacobian captures how each layer transforms the gradient signal during backpropagation, letting you apply the multivariable chain rule efficiently across the entire network.

Key Takeaway: The Jacobian matrix organizes all partial derivatives of a vector-valued function into rows (one per output) and columns (one per input), forming the foundation for gradient flow in neural networks.