Epsilon-Greedy Action Selection
What You'll Learn
You'll learn how to implement epsilon-greedy exploration with decaying epsilon to balance exploration and exploitation during reinforcement learning training.
The Core Idea
Epsilon-greedy action selection is the practical bridge between theory and training in deep reinforcement learning. While you've learned the concept of epsilon-greedy before, now you're implementing it as a callable function that your DQN agent uses at every decision point.
Here's how it works: Given your current Q-network and a state, with probability ε (epsilon) you choose a random action (explore), and with probability 1-ε you choose the greedy action (exploit) — the one your Q-network thinks is best.
The implementation pattern looks like this:
def select_action(state, q_network, epsilon):
if random.random() < epsilon:
return random.randint(0, num_actions - 1) # Explore
else:
with torch.no_grad():
q_values = q_network(state)
return q_values.argmax().item() # Exploit
Decaying epsilon is crucial: you start with high exploration (ε ≈ 1.0) when your Q-network knows nothing, then gradually decrease epsilon (e.g., epsilon = max(epsilon_min, epsilon * decay_rate)) so the agent increasingly trusts its learned policy as training progresses.
Think of it like learning to drive: at first, you try everything (high ε). As you improve, you rely more on what you've learned, but still occasionally try new routes (low ε).
Implementation Note
Your select_action function bridges your Q-network and the environment, making the exploration-exploitation tradeoff concrete in every training step.
Key Takeaway: Epsilon-greedy action selection with decaying epsilon transforms exploration theory into practice by randomly balancing random actions (exploration) and greedy Q-network actions (exploitation), gradually shifting toward exploitation as training progresses.