Course contentsShow
← Machine Learning and Deep Learning
Lesson 2251 of 3,53849. Deep Reinforcement Learning: Policy-BasedFree lesson

Parameterized Policies

Representing policies as neural networks with continuous parameters that map states to action distributions.

Parameterized Policies

What you'll learn: How to represent policies as neural networks with learnable parameters that directly map states to probability distributions over actions.

The Core Idea

Until now, we've been learning action values (Q-functions) and deriving policies from them. But what if we could directly learn the policy itself? That's what parameterized policies do.

A parameterized policy uses a neural network with parameters θ (theta) to represent the function π_θ(a|s) — the probability of taking action a in state s. Instead of a lookup table or value function, your policy is now a differentiable function approximator with continuous parameters you can optimize using gradient descent.

How It Works

Think of the neural network as a translator: it takes a state as input and outputs a probability distribution over all possible actions. For discrete actions, the output layer typically uses a softmax activation to ensure valid probabilities (they sum to 1). For continuous actions, the network might output parameters of a Gaussian distribution (mean and variance).

The beauty here is that small changes to the network weights (parameters θ) create smooth, gradual changes in the policy's behavior. This is fundamentally different from value-based methods where small Q-value changes can cause abrupt policy switches.

Why This Matters

Parameterized policies enable:

  • Direct optimization of what you care about (the policy)
  • Stochastic policies that naturally handle exploration
  • Continuous action spaces where selecting "max Q" isn't feasible
  • Smooth policy updates that improve learning stability

Key Takeaway: Parameterized policies represent the policy as a neural network π_θ(a|s) with learnable parameters θ, enabling direct policy learning through gradient-based optimization rather than learning values first.