The Derivative Definition
What you'll learn: How to formally define the derivative using limits and understand it as an instantaneous rate of change.
What Is a Derivative?
The derivative measures how a function changes at a single point. Imagine you're tracking your position while driving—your speedometer shows your instantaneous speed, not just average speed over a trip. The derivative is exactly that: the instantaneous rate of change.
The Formal Definition
Building on limits (our previous concept), the derivative of a function f(x) at point x is defined as:
f'(x) = lim[h→0] (f(x + h) - f(x)) / h
Let's break this down:
- f(x + h) is the function value at a tiny step h away from x
- f(x + h) - f(x) is how much the function changed
- Divide by h gives the average rate of change over that interval
- Take the limit as h→0 to get the instantaneous rate
Intuition: Zooming In
Picture a curve on a graph. If you zoom out, it looks curved. But as you zoom in closer and closer to a single point, the curve starts to look like a straight line. The derivative is the slope of that line—the function's behavior right at that point.
Why This Matters in ML
In machine learning, we constantly ask: "If I change this parameter slightly, how much does my error change?" That's a derivative! Understanding this definition grounds everything from gradient descent to backpropagation.
Key Takeaway: The derivative is the limit of the rate of change as the interval shrinks to zero, giving us the instantaneous rate of change at a point—the foundation for optimizing models in machine learning.