Limits of message passing for node classification: How class-bottlenecks restrict signal-to-noise ratio

TL;DR

This paper analyzes how class-bottlenecks and heterophily limit message passing neural networks' ability to classify nodes, introducing a framework and graph rewiring method to improve performance across different graph structures.

Contribution

It provides a unifying statistical framework linking heterophily and bottlenecks via SNR, and introduces BRIDGE, a graph rewiring algorithm that enhances MPNN performance.

Findings

BRIDGE achieves near-perfect accuracy on synthetic benchmarks.

The framework offers diagnostic tools for MPNN performance.

Optimal structures for homophily are disjoint unions of single-class and bipartite clusters.

Abstract

Message passing neural networks (MPNNs) are powerful models for node classification but suffer from performance limitations under heterophily (low same-class connectivity) and structural bottlenecks in the graph. We provide a unifying statistical framework exposing the relationship between heterophily and bottlenecks through the signal-to-noise ratio (SNR) of MPNN representations. The SNR decomposes model performance into feature-dependent parameters and feature-independent sensitivities. We prove that the sensitivity to class-wise signals is bounded by higher-order homophily -- a generalisation of classical homophily to multi-hop neighbourhoods -- and show that low higher-order homophily manifests locally as the interaction between structural bottlenecks and class labels (class-bottlenecks). Through analysis of graph ensembles, we provide a further quantitative decomposition of bottlenecking into underreaching (lack of depth implying signals cannot arrive) and oversquashing (lack of breadth implying signals arriving on fewer paths) with closed-form expressions. We prove that optimal graph structures for maximising higher-order homophily are disjoint unions of single-class and two-class-bipartite clusters. This yields BRIDGE, a graph ensemble-based rewiring algorithm that achieves near-perfect classification accuracy across all homophily regimes on synthetic benchmarks and significant improvements on real-world benchmarks, by eliminating the ``mid-homophily pitfall'' where MPNNs typically struggle, surpassing current standard rewiring techniques from the literature. Our framework, whose code we make available for public use, provides both diagnostic tools for assessing MPNN performance, and simple yet effective methods for enhancing performance through principled graph modification.