Course contentsShow
← Machine Learning and Deep Learning
Lesson 3268 of 3,53871. Interpretability: Neural Network MethodsFree lesson

Feature Visualization and Neuron Analysis

Techniques for understanding what individual neurons detect by generating inputs that maximally activate them.

Feature Visualization and Neuron Analysis

What you'll learn: How to reverse-engineer what individual neurons "see" by generating images that maximally activate them.

The Core Idea

Rather than feeding inputs through a network and seeing what fires, feature visualization works backward: it creates synthetic inputs that make a specific neuron light up as strongly as possible. Think of it like asking "what would my perfect dream meal look like?" to a food critic's brain cell—you're generating the ideal stimulus that triggers maximum excitement.

This reveals what patterns, textures, or concepts a neuron has learned to detect during training.

How It Works

The process inverts the usual forward pass:

  1. Pick a target neuron at any layer (e.g., neuron 47 in conv layer 3)
  2. Start with random noise as an input image
  3. Optimize the input (not the weights!) to maximize that neuron's activation
  4. Use gradient ascent: compute gradients of the neuron's output with respect to the input pixels
  5. Update the pixels to increase activation, adding regularization to keep images interpretable

What We Discover

Early layers often visualize simple patterns: edges, colors, or textures. Middle layers detect more complex structures like corners or simple shapes. Deep layers respond to high-level concepts—faces, wheels, or even specific object parts.

This technique connects directly to circuits and features analysis: once you know what a neuron detects, you can trace how it connects to others to form computational circuits.

Key Takeaway: Feature visualization generates synthetic inputs that maximally activate specific neurons, revealing what patterns they've learned to detect—a powerful tool for understanding the internal representations of neural networks.