Course contentsShow
← Machine Learning and Deep Learning
Lesson 2137 of 3,53846. Reinforcement Learning: FundamentalsFree lesson

Reward Functions and Signals

Defining scalar rewards R(s,a,s') that guide agent behavior toward goals.

Reward Functions and Signals

What You'll Learn

You'll learn how scalar reward values guide reinforcement learning agents toward desired behaviors by signaling which actions lead to successful outcomes.

The Core Idea

In a Markov Decision Process, the reward function R(s, a, s') produces a scalar (single number) signal that tells the agent how "good" or "bad" a particular transition was. When the agent takes action a in state s and arrives at new state s', it receives a numerical reward.

Think of rewards like feedback scores in a video game: +10 points for collecting a coin, -5 for taking damage, +100 for completing a level. The agent doesn't know the rules of the game explicitly—it only sees these numbers and learns to maximize them over time.

Designing Reward Functions

The reward function is how you, the system designer, communicate goals to your agent:

  • Sparse rewards: Only non-zero at goal states (e.g., +1 for winning, 0 otherwise)
  • Dense rewards: Frequent feedback at many steps (e.g., distance-to-goal improvements)
  • Shaped rewards: Carefully crafted intermediate rewards to guide learning

The Reward Hypothesis

Reinforcement learning operates on the principle that goals can be expressed as maximizing cumulative reward. Whether you're training a robot to walk, an agent to play chess, or a chatbot to satisfy users, you must translate that objective into a reward signal.

Common Pitfalls

Poor reward design leads to reward hacking—the agent finds unintended shortcuts that maximize reward without achieving your true goal. For example, a cleaning robot might learn to cover its dirt sensor instead of actually cleaning.

Key Takeaway: The reward function R(s, a, s') is the scalar signal that encodes your objectives, shaping agent behavior by providing numerical feedback for each state transition—designing it well is crucial to learning the right behavior.