Course contentsShow
← Machine Learning and Deep Learning
Lesson 2218 of 3,53848. Deep Reinforcement Learning: Value-BasedFree lesson

DQN Hyperparameters and Tuning

Critical hyperparameters: learning rate, batch size, replay size, target update frequency, and their effects.

DQN Hyperparameters and Tuning

What you'll learn: How to identify and tune the critical hyperparameters that determine DQN's learning stability and performance.

Why Hyperparameters Matter

Deep Q-Networks are sensitive to configuration choices. The wrong settings can lead to unstable learning, slow convergence, or complete failure to learn. Understanding which knobs to turn—and why—is essential for successful training.

Critical Hyperparameters

Learning Rate

Controls how aggressively the Q-network updates its weights. Too high causes oscillation and instability; too low means painfully slow learning. Typical values: 1e-4 to 1e-3. Think of it as your step size when climbing down a foggy mountain—too big and you overshoot valleys, too small and you'll never reach the bottom.

Batch Size

Determines how many experiences you sample from the replay buffer per training step. Larger batches (e.g., 32 to 128) provide more stable gradients but require more computation. Smaller batches update faster but with noisier gradients.

Replay Buffer Size

How many past experiences you store. Larger buffers (1e5 to 1e6) break correlation better and provide diverse training data, but consume more memory. Too small and you'll overfit to recent experiences.

Target Network Update Frequency

How often you sync the target network with the online network. Update too frequently (every few steps) and you lose stability; too rarely (every few thousand steps) and your targets become stale. Typical range: every 1000 to 10000 steps or using soft updates with tau=0.001.

Tuning Strategy

Start with proven defaults from research papers, then adjust one parameter at a time. Monitor loss curves and episode rewards. If training diverges, reduce learning rate or increase target update frequency. If learning stalls, check exploration (epsilon schedule) and replay buffer diversity.

Key Takeaway: DQN success hinges on balancing learning rate, batch size, replay buffer capacity, and target update frequency—each affects stability, sample efficiency, and convergence speed in interconnected ways.