Course contentsShow
← Machine Learning and Deep Learning
Lesson 2134 of 3,53846. Reinforcement Learning: FundamentalsFree lesson

States, Actions, and State Spaces

Understanding the components: state space S, action space A, and how agents interact with environments.

States, Actions, and State Spaces

What you'll learn: How agents perceive their environment through states, what actions they can take, and what these concepts mean in reinforcement learning.

The Core Building Blocks

In reinforcement learning, an agent interacts with an environment through a simple but powerful cycle. To understand this interaction, you need three fundamental components:

State (s)

A state is a snapshot of the environment at a specific moment. It contains all the information the agent needs to make a decision. Think of it like the configuration of a chessboard—where every piece sits tells you everything about the current game situation.

Action (a)

An action is a choice the agent can make. Actions change the state of the environment. In chess, moving a pawn forward is an action. In a self-driving car, turning the steering wheel left is an action.

State Space (S)

The state space is the set of all possible states the environment can be in. For tic-tac-toe, this includes every possible board configuration (empty board, one X placed, two Os and one X, etc.). State spaces can be:

  • Discrete: Countable states (chess positions, grid cells)
  • Continuous: Infinite states (robot arm angles, car position coordinates)

Action Space (A)

Similarly, the action space is the set of all possible actions available to the agent. This can also be discrete (move up/down/left/right) or continuous (accelerate from 0-100% throttle).

Why This Matters

These components define the "language" of how agents interact with their world. A well-defined state space captures what the agent can observe, and the action space defines what it can do about it.

Key Takeaway: States represent "where you are," actions represent "what you can do," and state/action spaces define all possibilities—the fundamental vocabulary for reinforcement learning problems.