Course contentsShow
← Machine Learning and Deep Learning
Lesson 3266 of 3,53871. Interpretability: Neural Network MethodsFree lesson

Circuits vs Features in Neural Networks

Distinguishing between individual features (neuron activations) and circuits (computational subgraphs that implement algorithms).

Circuits vs Features in Neural Networks

You'll learn how to distinguish between individual neuron activations (features) and the computational subgraphs that connect them (circuits).

The Core Distinction

In mechanistic interpretability, we separate two fundamental units of analysis:

Features are what individual neurons or activation patterns detect. Think of a single neuron in a vision model that fires when it sees a curve, or a particular direction in activation space that represents "dog-ness." Features are the vocabulary the network uses—individual meaningful elements.

Circuits are how features connect and compute together. A circuit is a subgraph of the network—specific neurons and weights working together to implement an algorithm. For example, detecting edges (feature A), combining them into curves (feature B), and assembling curves into a wheel detector (feature C) forms a circuit. Circuits are the grammar—they show how features compose to solve tasks.

Why This Matters

Studying features alone is like knowing individual words but not sentences. A neuron might detect "striped pattern," but the circuit reveals how the network uses that stripe detector in combination with "orange color" and "face shape" to identify tigers.

Circuits reveal:

  • Algorithmic structure: What computation the network actually performs
  • Compositional hierarchy: How simple features build complex ones
  • Causal pathways: Which connections matter for specific behaviors

Key Takeaway: Features are individual detectors (neurons or directions in activation space), while circuits are the computational subgraphs connecting features to implement algorithms—understanding both is essential for mechanistic interpretability.