Course contentsShow
← Machine Learning and Deep Learning
Lesson 2140 of 3,53846. Reinforcement Learning: FundamentalsFree lesson

Policies: Deterministic vs Stochastic

Representing agent behavior as mappings from states to actions: π(a|s) or π(s).

Policies: Deterministic vs Stochastic

What you'll learn: How to represent an agent's behavior as a policy that maps states to actions, either with certainty or probability.

The Core Idea

A policy is the strategy your agent follows—it tells the agent what action to take in any given state. Think of it as the agent's "rulebook" or "decision-making brain."

Policies come in two flavors:

Deterministic Policies: π(s)

A deterministic policy always chooses the same action for a given state. It's a direct mapping: π(s) = a.

Analogy: Like a vending machine—press button B3, and you always get the same snack. No randomness, no surprises.

When you're in state s, a deterministic policy outputs exactly one action. For example:

  • State: "Low health in game" → Action: "Use health potion" (always)

Stochastic Policies: π(a|s)

A stochastic policy defines a probability distribution over actions for each state. It's written as π(a|s), read as "the probability of taking action a given state s."

Analogy: Like choosing your lunch strategy—when hungry (state), you might pick pizza 60% of the time, salad 30%, and skip 10%. There's randomness built in.

For state s, a stochastic policy assigns probabilities to all possible actions, and they sum to 1.0.

Why Stochastic Policies?

Stochastic policies are useful for:

  • Exploration: Trying different actions to discover better strategies
  • Optimal strategies in some games: Sometimes mixing actions probabilistically is actually optimal (like bluffing in poker)
  • Expressiveness: More flexible for learning complex behaviors

Key Takeaway: A policy π represents how your agent behaves—deterministic policies output one action per state π(s), while stochastic policies output a probability distribution over actions π(a|s).