Course contentsShow
← Machine Learning and Deep Learning
Lesson 2502 of 3,53854. Graph Neural NetworksFree lesson

Normalization in Graph Convolutions

Learn why and how symmetric normalization using degree matrices stabilizes graph convolution operations.

Normalization in Graph Convolutions

What you'll learn: How symmetric normalization using degree matrices prevents exploding or vanishing signals during graph convolution operations.

The Problem: Unstable Signal Propagation

When aggregating messages from neighbors in graph convolutions, nodes with many connections (high degree) receive much larger summed signals than isolated nodes. Without normalization, repeated convolution layers cause these signals to explode for high-degree nodes and vanish for low-degree ones—making training impossible.

Why Symmetric Normalization?

Symmetric normalization scales messages by both the sender's and receiver's degrees. For an edge from node i to node j, we multiply the message by:

1 / sqrt(degree_i × degree_j)

This appears in the GCN layer formula as D^(-½) A D^(-½), where:

  • D is the diagonal degree matrix
  • A is the adjacency matrix
  • The operation "sandwiches" A between inverse square-root degree matrices

Intuition: Balanced Information Flow

Think of degree normalization like volume control in a conversation network. A celebrity (high-degree node) broadcasting to thousands shouldn't drown out a quiet dialogue between two friends (low-degree nodes). Symmetric normalization ensures each connection contributes proportionally, regardless of how "popular" the nodes are.

The square root split (½ from sender, ½ from receiver) maintains mathematical symmetry, which preserves important spectral properties of the graph Laplacian and ensures gradients flow stably during backpropagation.

Key Result

Without this normalization, the eigenvalues of the unnormalized adjacency matrix can range wildly, causing gradient instability. Symmetric normalization constrains eigenvalues to [-1, 1], enabling stable, deep graph neural networks.

Key Takeaway: Symmetric normalization using degree matrices (D^(-½) A D^(-½)) stabilizes graph convolutions by balancing message contributions from high- and low-degree nodes, enabling effective multi-layer learning.