Course contentsShow
← Machine Learning and Deep Learning
Lesson 2277 of 3,53849. Deep Reinforcement Learning: Policy-BasedFree lesson

The Actor: Parameterized Policy Networks

Define how the actor network represents and outputs action probabilities or continuous action values.

The Actor: Parameterized Policy Networks

What you'll learn: How the actor network transforms states into action probabilities or continuous values using learnable parameters.

What Is the Actor?

In actor-critic methods, the actor is a neural network that directly represents your policy—it's the component that decides what action to take given a state. Unlike value-based methods (like DQN) that learn Q-values and derive actions indirectly, the actor explicitly outputs actions or action distributions.

How the Actor Network Works

The actor takes the current state as input and produces outputs that depend on your action space:

For Discrete Actions

The network outputs a probability distribution over all possible actions using a softmax activation on the final layer. Each output neuron corresponds to one action, and the softmax ensures probabilities sum to 1.

Analogy: Think of it like a weather forecaster giving probabilities: 60% chance of rain (action 1), 30% sunny (action 2), 10% snow (action 3).

For Continuous Actions

The network outputs parameters of a continuous distribution—typically the mean (and sometimes variance) of a Gaussian distribution. You then sample from this distribution to get the actual action value.

Analogy: Like a thermostat controller that outputs "set temperature to 72°F ± 2°" rather than choosing from preset options.

Why Parameterization Matters

By using neural network weights as learnable parameters (θ), we can represent complex policies that map high-dimensional states to appropriate actions. The policy gradient theorem tells us how to update these parameters to improve expected return.

Key Takeaway: The actor network parameterizes your policy π(a|s; θ), outputting action probabilities for discrete spaces or distribution parameters for continuous spaces, making the policy differentiable and optimizable through gradient ascent.