Evaluation and Testing Protocol
What You'll Learn
You'll learn how to properly evaluate your trained DQN agent by turning off exploration and running deterministic test episodes to measure true learned performance.
The Core Idea
During training, your agent uses epsilon-greedy exploration to discover new strategies—sometimes taking random actions to learn. But when evaluating how well your agent has actually learned, you want to see its best performance without any random experimentation.
Evaluation mode means setting epsilon=0, so your agent always takes the action it believes is best (the greedy action with highest Q-value). You then run multiple test episodes and average the rewards to get a reliable measure of your policy's quality.
Think of it like a student studying for an exam: during practice (training), they explore different problem-solving approaches, sometimes trying unconventional methods. But during the actual test (evaluation), they use only their best-learned strategies.
Implementation Pattern
def evaluate_agent(env, q_network, num_episodes=10, epsilon=0.0):
"""Evaluate agent performance without exploration"""
total_rewards = []
for episode in range(num_episodes):
state = env.reset()
episode_reward = 0
done = False
while not done:
# Greedy action selection (epsilon=0)
q_values = q_network(state)
action = q_values.argmax()
state, reward, done, _ = env.step(action)
episode_reward += reward
total_rewards.append(episode_reward)
return sum(total_rewards) / len(total_rewards)
Run evaluation periodically during training (e.g., every 10,000 steps) to track learning progress without the noise of exploration.
Key Takeaway: Proper evaluation requires epsilon=0 (greedy action selection) over multiple test episodes to measure your agent's true learned performance, separate from training exploration.