Course contentsShow
← Machine Learning and Deep Learning
Lesson 2241 of 3,53848. Deep Reinforcement Learning: Value-BasedFree lesson

The Environment Interaction Loop

Write the agent-environment interaction code: taking actions, observing rewards, storing transitions, and episode management.

The Environment Interaction Loop

What you'll learn: How to implement the core loop where your DQN agent takes actions, observes outcomes, and learns from experience.

The Heart of RL Training

The environment interaction loop is where everything comes together. Think of it as a conversation between your agent and the world: the agent speaks (takes action), the world responds (returns state and reward), and the agent remembers (stores the experience). This cycle repeats thousands or millions of times during training.

The Four-Step Dance

Here's the fundamental pattern:

state = env.reset()
done = False
episode_reward = 0

while not done:
    # 1. Agent selects action (epsilon-greedy)
    action = select_action_epsilon_greedy(state, q_network, epsilon)
    
    # 2. Environment responds
    next_state, reward, done, info = env.step(action)
    
    # 3. Store transition in replay buffer
    replay_buffer.add(state, action, reward, next_state, done)
    
    # 4. Train agent (if enough samples)
    if len(replay_buffer) > batch_size:
        batch = replay_buffer.sample(batch_size)
        loss = train_step(batch, q_network, target_network)
    
    # Update for next iteration
    state = next_state
    episode_reward += reward

print(f"Episode finished with reward: {episode_reward}")

Episode Management

Each episode starts with env.reset() and continues until done=True signals termination. You'll typically track episode rewards, lengths, and other metrics to monitor training progress. Between episodes, you might decay epsilon or update the target network.

Key Takeaway: The environment interaction loop orchestrates action selection, environment stepping, experience storage, and training—repeating this cycle builds the dataset your DQN learns from while simultaneously improving the policy.