Epsilon-Greedy Action Selection
What you'll learn: How to balance exploiting your best-known action with random exploration using a simple probability threshold called epsilon.
The Core Idea
Epsilon-greedy action selection is a practical strategy for solving the exploration-exploitation dilemma in reinforcement learning. With probability ε (epsilon), you choose a random action to explore new possibilities. With probability (1 - ε), you exploit your current knowledge by selecting the action with the highest estimated value.
Think of it like choosing restaurants: most of the time (say, 90%), you go to your favorite spot (exploitation), but occasionally (10%), you try somewhere new to see if there's something even better (exploration).
How It Works
Given epsilon (typically between 0 and 1):
- Generate a random number between 0 and 1
- If the number < ε: Choose a random action uniformly from all available actions
- Otherwise: Choose the greedy action (the one with highest Q-value or action-value estimate)
For example, with ε = 0.1:
- 10% of the time: explore randomly
- 90% of the time: exploit the best-known action
Why It Matters
In the Multi-Armed Bandit Problem and other RL settings, pure greedy selection (always picking the apparent best action) can trap you in suboptimal behavior—you never discover that another arm might be better. Epsilon-greedy ensures you periodically check alternatives, gradually building a more accurate picture of each action's true value.
The method is simple to implement, computationally cheap, and surprisingly effective across many domains.
Key Takeaway: Epsilon-greedy balances exploration and exploitation by acting randomly with probability ε and greedily with probability (1 - ε), ensuring you both discover new information and leverage what you already know.