Course contentsShow
← Machine Learning and Deep Learning
Lesson 2187 of 3,53847. Reinforcement Learning: Temporal Difference MethodsFree lesson

Epsilon-Greedy Exploration

The most common exploration strategy: choosing random actions with probability ε and greedy actions otherwise.

Epsilon-Greedy Exploration

What you'll learn: How to balance exploration and exploitation using a simple probability-based strategy that randomly explores with probability ε.

The Core Idea

Epsilon-greedy is the most widely used exploration strategy in reinforcement learning. The concept is beautifully simple: with probability ε (epsilon), choose a random action to explore; otherwise (with probability 1-ε), choose the greedy action that currently looks best according to your Q-values.

Think of it like trying restaurants: most of the time (say, 90%) you go to your favorite place, but occasionally (10%) you randomly pick a new restaurant to try—you might discover something even better!

How It Works

At each step, you:

  1. Generate a random number between 0 and 1
  2. If it's less than ε: pick a random action (explore)
  3. Otherwise: pick the action with the highest Q-value (exploit)

Typical values: ε = 0.1 (explore 10% of the time) or ε = 0.05 (explore 5%)

Why It Matters

Remember from "Greedy Action Selection and Its Limitations" that pure greedy selection gets stuck in local optima. Epsilon-greedy solves this by guaranteeing continued exploration, ensuring every state-action pair gets visited infinitely often (in theory), which is necessary for Q-learning and SARSA to converge to optimal policies.

The beauty is its simplicity: just one parameter controls the exploration-exploitation trade-off. Too high? You waste time on bad actions. Too low? You might never discover better strategies.

Key Takeaway: Epsilon-greedy exploration randomly explores with probability ε and exploits greedily otherwise, providing a simple yet effective solution to the exploration-exploitation dilemma that works well in practice for TD methods like Q-learning and SARSA.