Course contentsShow
← Machine Learning and Deep Learning
Lesson 2696 of 3,53858. Model Compression: Pruning and DistillationFree lesson

Reinforcement Learning for NAS

Using RL agents to generate architectures as sequences of decisions, rewarded by validation performance.

Reinforcement Learning for NAS

What You'll Learn

You'll learn how reinforcement learning agents can automatically design neural network architectures by treating architecture choices as sequential decisions rewarded by performance.

The Core Idea

Instead of manually choosing layers, filters, and connections, imagine an RL agent as an architect-in-training. At each step, the agent makes a decision: "Should I add a convolutional layer or a pooling layer next? How many filters?" Each complete sequence of decisions produces a full neural network architecture.

The agent then trains this generated network on your task and observes its validation accuracy. This accuracy becomes the reward signal. High accuracy? The agent learns those architectural choices were good. Low accuracy? It learns to avoid similar patterns.

How It Works

The RL agent (often using policy gradient methods) operates in a loop:

  1. State: Current partial architecture
  2. Action: Choose next layer type, hyperparameter, or connection
  3. Policy: The agent's learned strategy for making architecture decisions
  4. Reward: Validation performance of the completed network
  5. Update: Adjust the policy to favor high-reward architecture sequences

This process treats NAS as a sequential decision problem. The agent explores the search space intelligently, gradually learning which architectural patterns yield better performance than random or grid search would discover.

The challenge? Training each candidate architecture is expensive, so the agent must learn efficiently from relatively few architecture evaluations.

Why This Matters for Model Compression

RL-based NAS can discover compact, efficient architectures directly—networks that balance accuracy with size or speed constraints, which is exactly what pruning and distillation aim for post-hoc.

Key Takeaway: Reinforcement learning transforms NAS into a sequential decision problem where an agent learns to generate high-performing architectures by receiving validation accuracy as reward signals.