Basic Differentiation Rules
What you'll learn: How to compute derivatives quickly using three fundamental rules that turn calculus into simple algebra.
Why Rules Matter
Remember from "The Derivative Definition" that computing derivatives from first principles (using limits) works but is tedious. Fortunately, mathematicians discovered shortcuts—differentiation rules—that let you find derivatives almost instantly. These rules are essential for machine learning because we'll compute thousands of derivatives when training models.
The Three Core Rules
Power Rule
If you have a function like f(x) = x^n (x raised to any power), the derivative is:
f'(x) = n · x^(n-1)
Example: f(x) = x³ → f'(x) = 3x²
Think of it as "bring the power down front, then reduce the power by one."
Constant Rule
The derivative of any constant is zero:
f'(x) = 0 when f(x) = c
Example: f(x) = 7 → f'(x) = 0
This makes sense geometrically—a constant function is a flat horizontal line (slope = 0).
Sum Rule
The derivative of a sum is the sum of the derivatives:
If f(x) = g(x) + h(x), then f'(x) = g'(x) + h'(x)
Example: f(x) = x² + 5x → f'(x) = 2x + 5
You can differentiate each term separately and add the results.
Combining the Rules
These rules work together beautifully:
f(x) = 3x⁴ + 2x² + 8
- Term 1: 3x⁴ → 12x³ (power rule)
- Term 2: 2x² → 4x (power rule)
- Term 3: 8 → 0 (constant rule)
- Result: f'(x) = 12x³ + 4x (sum rule)
Key Takeaway: The power, constant, and sum rules transform derivative calculations from limit-based tedium into fast algebraic operations—exactly what we need when optimizing machine learning models.