Course contentsShow
← Machine Learning and Deep Learning
Lesson 31 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

The Derivative Definition

Deriving the derivative from first principles using limits and interpreting it as instantaneous rate of change.

The Derivative Definition

What you'll learn: How to formally define the derivative using limits and understand it as an instantaneous rate of change.

What Is a Derivative?

The derivative measures how a function changes at a single point. Imagine you're tracking your position while driving—your speedometer shows your instantaneous speed, not just average speed over a trip. The derivative is exactly that: the instantaneous rate of change.

The Formal Definition

Building on limits (our previous concept), the derivative of a function f(x) at point x is defined as:

f'(x) = lim[h→0] (f(x + h) - f(x)) / h

Let's break this down:

  • f(x + h) is the function value at a tiny step h away from x
  • f(x + h) - f(x) is how much the function changed
  • Divide by h gives the average rate of change over that interval
  • Take the limit as h→0 to get the instantaneous rate

Intuition: Zooming In

Picture a curve on a graph. If you zoom out, it looks curved. But as you zoom in closer and closer to a single point, the curve starts to look like a straight line. The derivative is the slope of that line—the function's behavior right at that point.

Why This Matters in ML

In machine learning, we constantly ask: "If I change this parameter slightly, how much does my error change?" That's a derivative! Understanding this definition grounds everything from gradient descent to backpropagation.

Key Takeaway: The derivative is the limit of the rate of change as the interval shrinks to zero, giving us the instantaneous rate of change at a point—the foundation for optimizing models in machine learning.