The Environment Interaction Loop
What you'll learn: How to implement the core loop where your DQN agent takes actions, observes outcomes, and learns from experience.
The Heart of RL Training
The environment interaction loop is where everything comes together. Think of it as a conversation between your agent and the world: the agent speaks (takes action), the world responds (returns state and reward), and the agent remembers (stores the experience). This cycle repeats thousands or millions of times during training.
The Four-Step Dance
Here's the fundamental pattern:
state = env.reset()
done = False
episode_reward = 0
while not done:
# 1. Agent selects action (epsilon-greedy)
action = select_action_epsilon_greedy(state, q_network, epsilon)
# 2. Environment responds
next_state, reward, done, info = env.step(action)
# 3. Store transition in replay buffer
replay_buffer.add(state, action, reward, next_state, done)
# 4. Train agent (if enough samples)
if len(replay_buffer) > batch_size:
batch = replay_buffer.sample(batch_size)
loss = train_step(batch, q_network, target_network)
# Update for next iteration
state = next_state
episode_reward += reward
print(f"Episode finished with reward: {episode_reward}")
Episode Management
Each episode starts with env.reset() and continues until done=True signals termination. You'll typically track episode rewards, lengths, and other metrics to monitor training progress. Between episodes, you might decay epsilon or update the target network.
Key Takeaway: The environment interaction loop orchestrates action selection, environment stepping, experience storage, and training—repeating this cycle builds the dataset your DQN learns from while simultaneously improving the policy.