Course contentsShow
← Machine Learning and Deep Learning
Lesson 30 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

Limits: The Foundation of Derivatives

Defining limits rigorously and understanding how they capture instantaneous rate of change.

Limits: The Foundation of Derivatives

What you'll learn: How limits capture the precise moment-by-moment change in a function—the key to understanding derivatives.

What Is a Limit?

A limit describes what value a function approaches as its input gets arbitrarily close to some point. Think of it like zooming in on a curve with a magnifying glass: as you focus on a tiny neighborhood around a point, the function's behavior becomes clearer.

Formally, we say "the limit of f(x) as x approaches a equals L" if we can make f(x) as close to L as we want by choosing x sufficiently close to a (but not necessarily equal to a).

Why Limits Matter for Derivatives

Imagine you're tracking a car's position over time. Its average speed between two moments is straightforward: distance divided by time. But what about its instantaneous speed at one exact moment?

This is where limits shine. To find instantaneous rate of change:

  1. Measure the average change over a tiny interval
  2. Shrink that interval toward zero
  3. The limit of these shrinking average changes gives you the instantaneous rate

This process—taking the limit of average rates as the interval shrinks—is the derivative. In machine learning, derivatives tell us how loss functions change when we adjust model parameters, guiding optimization algorithms like gradient descent.

The Connection to Continuity

From our previous lesson on continuity, you know a function is continuous if small input changes produce small output changes. Limits formalize this: a function is continuous at a point if the limit equals the function's actual value there. Derivatives require this foundation—you can't measure smooth change if the function jumps around.

Key Takeaway: Limits rigorously define what happens "in the infinitesimal," allowing us to capture instantaneous rates of change—the essence of derivatives and the mathematics powering gradient-based learning.