Course contentsShow
← Machine Learning and Deep Learning
Lesson 36 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

Derivatives of Exponential Functions

Finding derivatives of exponential functions, particularly e^x, crucial for activation functions.

Derivatives of Exponential Functions

What you'll learn: How to find the derivative of exponential functions, especially e^x, which appears constantly in neural network activation functions.

Why Exponentials Matter in ML

Exponential functions pop up everywhere in machine learning—sigmoid activations, softmax outputs, and probability distributions all rely on e^x. To train neural networks effectively, you need to compute gradients, which means taking derivatives of these exponential functions.

The Core Idea

The exponential function e^x has a beautiful mathematical property: its derivative is itself.

If f(x) = e^x, then f'(x) = e^x

This means the rate of change of e^x at any point equals the function's value at that point. If e^2 ≈ 7.39, then the slope of the curve at x=2 is also 7.39.

Why This Happens

Think of exponential growth like compound interest: the more you have, the faster it grows. The derivative measures "how fast," and for e^x, that growth rate is proportional to the current amount—they're exactly equal.

Chain Rule Extension

When you have e^(something), use the chain rule you learned earlier:

If f(x) = e^(g(x)), then f'(x) = e^(g(x)) × g'(x)

Example: For f(x) = e^(2x):

  • Outer function: e^u where u = 2x
  • Inner function derivative: g'(x) = 2
  • Result: f'(x) = e^(2x) × 2 = 2e^(2x)

Other Exponential Bases

For a general base a^x (like 2^x or 10^x):

  • Derivative: a^x × ln(a)

The natural base e is special because ln(e) = 1, making the derivative simplest.

Key Takeaway: The derivative of e^x is e^x itself—this unique property makes exponential functions computationally elegant in neural networks, where you'll repeatedly compute gradients during backpropagation.