Course contentsShow
← Machine Learning and Deep Learning
Lesson 2248 of 3,53848. Deep Reinforcement Learning: Value-BasedFree lesson

Evaluation and Testing Protocol

Implement evaluation mode with epsilon=0, running multiple test episodes to assess learned policy performance without exploration.

Evaluation and Testing Protocol

What You'll Learn

You'll learn how to properly evaluate your trained DQN agent by turning off exploration and running deterministic test episodes to measure true learned performance.

The Core Idea

During training, your agent uses epsilon-greedy exploration to discover new strategies—sometimes taking random actions to learn. But when evaluating how well your agent has actually learned, you want to see its best performance without any random experimentation.

Evaluation mode means setting epsilon=0, so your agent always takes the action it believes is best (the greedy action with highest Q-value). You then run multiple test episodes and average the rewards to get a reliable measure of your policy's quality.

Think of it like a student studying for an exam: during practice (training), they explore different problem-solving approaches, sometimes trying unconventional methods. But during the actual test (evaluation), they use only their best-learned strategies.

Implementation Pattern

def evaluate_agent(env, q_network, num_episodes=10, epsilon=0.0):
    """Evaluate agent performance without exploration"""
    total_rewards = []
    
    for episode in range(num_episodes):
        state = env.reset()
        episode_reward = 0
        done = False
        
        while not done:
            # Greedy action selection (epsilon=0)
            q_values = q_network(state)
            action = q_values.argmax()
            
            state, reward, done, _ = env.step(action)
            episode_reward += reward
        
        total_rewards.append(episode_reward)
    
    return sum(total_rewards) / len(total_rewards)

Run evaluation periodically during training (e.g., every 10,000 steps) to track learning progress without the noise of exploration.

Key Takeaway: Proper evaluation requires epsilon=0 (greedy action selection) over multiple test episodes to measure your agent's true learned performance, separate from training exploration.