Course contentsShow
← Machine Learning and Deep Learning
Lesson 2252 of 3,53849. Deep Reinforcement Learning: Policy-BasedFree lesson

Stochastic vs Deterministic Policies

Understanding when to use probability distributions over actions versus direct action mappings.

Stochastic vs Deterministic Policies

You'll learn when to output probabilities over actions versus selecting a single definite action.

The Core Distinction

In policy-based reinforcement learning, your parameterized policy can make decisions in two fundamentally different ways:

A deterministic policy directly maps states to specific actions: given state s, always take action a. It's like a GPS giving you one clear direction: "Turn left."

A stochastic policy outputs a probability distribution over actions: given state s, sample from a distribution over possible actions. It's like asking a local for directions and getting: "Turn left 70% of the time, or try the shortcut 30% of the time."

When to Use Each

Deterministic policies work well when:

  • Your environment is deterministic and fully observable
  • You want efficient exploitation after learning
  • Continuous action spaces with clear optimal actions (e.g., robotic control)

Stochastic policies are essential when:

  • You need built-in exploration during training
  • The optimal strategy genuinely requires randomness (rock-paper-scissors, poker)
  • Dealing with partial observability or aliased states
  • You want the policy gradient theorem to apply cleanly (stochastic policies provide natural gradients)

The Policy-Based Advantage

Stochastic policies are particularly powerful in policy gradient methods because they enable gradient-based learning through probability distributions. Instead of learning which action is best, you learn how to adjust the probabilities of each action.

Key Takeaway: Deterministic policies pick one action per state; stochastic policies sample from action probabilities. Use stochastic policies for exploration and policy gradients, deterministic for efficient execution in well-understood environments.