Course contentsShow
← Machine Learning and Deep Learning
Lesson 3271 of 3,53871. Interpretability: Neural Network MethodsFree lesson

Path Patching and Circuit Discovery

Identifying computational pathways by patching specific edges in the network's computation graph to isolate circuits.

Path Patching and Circuit Discovery

What you'll learn: How to isolate specific computational pathways in neural networks by surgically patching individual edges in the computation graph.

The Core Idea

While activation patching lets you intervene on entire neurons or layers, path patching takes a more precise scalpel: it targets individual connections (edges) between components. Think of a neural network as a city's road network. Activation patching closes entire intersections, but path patching closes specific roads to see which routes traffic actually uses to get from point A to point B.

The technique works by replacing the signal flowing along a specific edge (say, from attention head 3.2 to MLP layer 4) on a corrupted input with the signal from a clean input. If restoring that single path recovers model performance, you've found a critical wire in the circuit.

Why This Matters for Circuit Discovery

Path patching reveals the minimal computational circuit needed for a task. By systematically patching edges:

  1. You identify which attention heads connect to which MLPs
  2. You discover which layer-to-layer connections matter
  3. You filter out thousands of unused or redundant pathways
  4. You can draw a clean circuit diagram showing only the essential data flow

This is more granular than activation patching because you're not asking "does layer 5 matter?" but rather "does the specific connection from head 2.1 to neuron 378 in layer 6 matter?"

The result is a sparse graph of the actual algorithm the network implements—stripping away the noise to reveal the core computation.

Key Takeaway: Path patching isolates individual edges in the computation graph, enabling precise discovery of the minimal circuits networks use to perform specific tasks.