Course contentsShow
← Machine Learning and Deep Learning
Lesson 2505 of 3,53854. Graph Neural NetworksFree lesson

Multiple Message Passing Layers and Depth

Understand how stacking layers expands receptive fields and the challenges of over-smoothing in deep GNNs.

Multiple Message Passing Layers and Depth

What you'll learn: How stacking multiple message passing layers expands a node's receptive field, and why going too deep can cause problems in Graph Neural Networks.

Why Stack Multiple Layers?

Just like convolutional layers in CNNs progressively capture larger spatial patterns, stacking message passing layers in GNNs allows nodes to gather information from increasingly distant neighbors.

With one layer, a node aggregates messages only from its immediate neighbors (1-hop away). With two layers, it receives information from neighbors' neighbors (2-hops). Each additional layer expands the receptive field by one hop.

Think of it like spreading news in a social network: after one round, your direct friends know; after two rounds, friends-of-friends hear about it; and so on.

The Over-Smoothing Problem

Here's the catch: as you stack more layers, node representations tend to become increasingly similar to each other. This phenomenon is called over-smoothing.

Why does this happen? Each aggregation step blends a node's features with its neighbors'. After many iterations, all nodes in a connected component converge toward the same averaged representation—losing the distinct information that makes each node unique.

Analogy: Imagine repeatedly averaging colors in adjacent regions of a painting. Eventually, everything becomes the same muddy gray.

This makes very deep GNNs (>4-8 layers) often perform worse than shallow ones, unlike vision models where depth consistently helps.

Practical Implications

  • Most effective GNNs use only 2-4 layers
  • Beyond that, consider skip connections, normalization techniques, or specialized architectures
  • The optimal depth depends on graph diameter and task requirements

Key Takeaway: Stacking message passing layers expands receptive fields hop-by-hop, but too many layers cause over-smoothing where node representations lose distinctiveness—limiting practical GNN depth to just a few layers.