Course contentsShow
← Machine Learning and Deep Learning
Lesson 2240 of 3,53848. Deep Reinforcement Learning: Value-BasedFree lesson

Epsilon-Greedy Action Selection

Implement epsilon-greedy exploration strategy with decaying epsilon for balancing exploration and exploitation during training.

Epsilon-Greedy Action Selection

What You'll Learn

You'll learn how to implement epsilon-greedy exploration with decaying epsilon to balance exploration and exploitation during reinforcement learning training.

The Core Idea

Epsilon-greedy action selection is the practical bridge between theory and training in deep reinforcement learning. While you've learned the concept of epsilon-greedy before, now you're implementing it as a callable function that your DQN agent uses at every decision point.

Here's how it works: Given your current Q-network and a state, with probability ε (epsilon) you choose a random action (explore), and with probability 1-ε you choose the greedy action (exploit) — the one your Q-network thinks is best.

The implementation pattern looks like this:

def select_action(state, q_network, epsilon):
    if random.random() < epsilon:
        return random.randint(0, num_actions - 1)  # Explore
    else:
        with torch.no_grad():
            q_values = q_network(state)
            return q_values.argmax().item()  # Exploit

Decaying epsilon is crucial: you start with high exploration (ε ≈ 1.0) when your Q-network knows nothing, then gradually decrease epsilon (e.g., epsilon = max(epsilon_min, epsilon * decay_rate)) so the agent increasingly trusts its learned policy as training progresses.

Think of it like learning to drive: at first, you try everything (high ε). As you improve, you rely more on what you've learned, but still occasionally try new routes (low ε).

Implementation Note

Your select_action function bridges your Q-network and the environment, making the exploration-exploitation tradeoff concrete in every training step.

Key Takeaway: Epsilon-greedy action selection with decaying epsilon transforms exploration theory into practice by randomly balancing random actions (exploration) and greedy Q-network actions (exploitation), gradually shifting toward exploitation as training progresses.