Course contentsShow
← Machine Learning and Deep Learning
Lesson 613 of 3,53815. Neural Network FoundationsFree lesson

Loss Functions: Purpose and Role in Training

Understanding how loss functions quantify prediction error and guide gradient descent to optimize neural network weights.

Loss Functions: Purpose and Role in Training

What you'll learn: How loss functions measure how wrong your neural network's predictions are and guide the training process to make better predictions.

What Is a Loss Function?

A loss function (also called a cost function or objective function) is a mathematical formula that quantifies how far off your model's predictions are from the actual target values. Think of it as a report card that gives your neural network a single score measuring its mistakes.

When your network performs forward propagation and produces predictions, the loss function compares those predictions to the true labels and outputs a number—the loss. A higher loss means worse predictions; a lower loss means better predictions.

Why Loss Functions Matter

Loss functions serve two critical purposes:

  1. Measuring Performance: They give you a concrete, numeric measure of your model's current accuracy. Without this, you wouldn't know if your network is improving or getting worse.

  2. Guiding Optimization: More importantly, the loss function provides the signal for gradient descent. During training, we calculate how the loss changes with respect to each weight in the network. These gradients tell us which direction to adjust each weight to reduce the loss—essentially teaching the network to make better predictions.

The Training Loop Connection

Here's the flow: forward propagation generates predictions → the loss function measures error → gradients are computed from this error → weights are updated to minimize future error. The loss function is the bridge between "what the network predicted" and "how to make it better."

Key Takeaway: Loss functions quantify prediction error as a single number and provide the gradient signal that drives neural network learning through weight optimization.