Product and Quotient Rules
You'll learn how to differentiate functions that are multiplied or divided—essential tools for handling complex loss functions in machine learning.
Why These Rules Matter
In machine learning, you often need derivatives of functions that combine simpler pieces through multiplication or division. For example, computing gradients of regularized loss functions or activation functions like sigmoid involves products and quotients.
The Product Rule
When you have two functions multiplied together, you can't just differentiate each separately. Instead:
If h(x) = f(x) · g(x), then h'(x) = f'(x) · g(x) + f(x) · g'(x)
Think of it like this: when both factors are changing, the overall rate of change comes from each factor's contribution—first function's rate times second function's current value, plus first function's current value times second function's rate.
Example: If h(x) = x² · sin(x), then:
- h'(x) = (2x) · sin(x) + x² · cos(x)
The Quotient Rule
When dividing functions, the rule is slightly more complex:
If h(x) = f(x) / g(x), then h'(x) = [f'(x) · g(x) - f(x) · g'(x)] / [g(x)]²
Memory trick: "low d-high minus high d-low, over low squared" (where "low" = denominator, "high" = numerator, "d" = derivative).
Example: If h(x) = x² / (x + 1), then:
- h'(x) = [2x · (x + 1) - x² · 1] / (x + 1)²
- h'(x) = [2x² + 2x - x²] / (x + 1)²
- h'(x) = [x² + 2x] / (x + 1)²
Key Takeaway: The product rule adds two terms (derivative of first times second, plus first times derivative of second), while the quotient rule uses the pattern "bottom times top's derivative minus top times bottom's derivative, all over bottom squared."