Implementing Basic Message Passing
What you'll learn: You'll code a simple message passing layer from scratch that aggregates neighbor features using adjacency matrices.
The Core Idea
Message passing is like a neighborhood gossip network: each node collects information from its neighbors, combines it, and updates its own understanding. In code, this means multiplying the adjacency matrix by the feature matrix to aggregate neighbor features.
From Theory to Implementation
Remember that the adjacency matrix A tells us who's connected to whom, and our feature matrix X holds each node's features. The simplest message passing operation is:
import torch
def simple_message_passing(adj_matrix, node_features):
"""
Aggregate neighbor features using adjacency matrix.
Args:
adj_matrix: (N, N) adjacency matrix
node_features: (N, F) node feature matrix
Returns:
(N, F) aggregated features
"""
# Matrix multiplication: each row of result is sum of neighbor features
aggregated = torch.mm(adj_matrix, node_features)
return aggregated
What's Happening?
When you compute A @ X, each node collects the features of all its neighbors and sums them. If node 0 connects to nodes 1 and 2, row 0 of the result contains the sum of features from nodes 1 and 2.
Adding Normalization
Raw aggregation can explode for high-degree nodes. Add degree normalization:
def normalized_message_passing(adj_matrix, node_features):
# Compute node degrees (row sums)
degree = adj_matrix.sum(dim=1, keepdim=True)
# Avoid division by zero
degree = torch.clamp(degree, min=1)
# Normalize: average instead of sum
normalized_adj = adj_matrix / degree
return torch.mm(normalized_adj, node_features)
Key Takeaway: Basic message passing is matrix multiplication between the adjacency matrix and node features—averaging neighbor information to update each node's representation.