Course contentsShow
← Machine Learning and Deep Learning
Lesson 2200 of 3,53847. Reinforcement Learning: Temporal Difference MethodsFree lesson

Epsilon-Greedy Action Selection

Balancing exploitation of best-known action with random exploration using probability epsilon.

Epsilon-Greedy Action Selection

What you'll learn: How to balance exploiting your best-known action with random exploration using a simple probability threshold called epsilon.

The Core Idea

Epsilon-greedy action selection is a practical strategy for solving the exploration-exploitation dilemma in reinforcement learning. With probability ε (epsilon), you choose a random action to explore new possibilities. With probability (1 - ε), you exploit your current knowledge by selecting the action with the highest estimated value.

Think of it like choosing restaurants: most of the time (say, 90%), you go to your favorite spot (exploitation), but occasionally (10%), you try somewhere new to see if there's something even better (exploration).

How It Works

Given epsilon (typically between 0 and 1):

  1. Generate a random number between 0 and 1
  2. If the number < ε: Choose a random action uniformly from all available actions
  3. Otherwise: Choose the greedy action (the one with highest Q-value or action-value estimate)

For example, with ε = 0.1:

  • 10% of the time: explore randomly
  • 90% of the time: exploit the best-known action

Why It Matters

In the Multi-Armed Bandit Problem and other RL settings, pure greedy selection (always picking the apparent best action) can trap you in suboptimal behavior—you never discover that another arm might be better. Epsilon-greedy ensures you periodically check alternatives, gradually building a more accurate picture of each action's true value.

The method is simple to implement, computationally cheap, and surprisingly effective across many domains.

Key Takeaway: Epsilon-greedy balances exploration and exploitation by acting randomly with probability ε and greedily with probability (1 - ε), ensuring you both discover new information and leverage what you already know.