Spectral Graph Pruning Against Over-Squashing and Over-Smoothing

TL;DR

This paper introduces a spectral gap-based graph pruning method that simultaneously mitigates over-squashing and over-smoothing in message passing graph neural networks, improving performance on large heterophilic datasets.

Contribution

It proposes a novel spectral gap optimization framework for edge addition and deletion, unifying solutions to over-squashing and over-smoothing in GNNs.

Findings

Edge deletions can improve GNN generalization.

Spectral gap optimization enhances GNN performance.

Effective on large heterophilic datasets.

Abstract

Message Passing Graph Neural Networks are known to suffer from two problems that are sometimes believed to be diametrically opposed: over-squashing and over-smoothing. The former results from topological bottlenecks that hamper the information flow from distant nodes and are mitigated by spectral gap maximization, primarily, by means of edge additions. However, such additions often promote over-smoothing that renders nodes of different classes less distinguishable. Inspired by the Braess phenomenon, we argue that deleting edges can address over-squashing and over-smoothing simultaneously. This insight explains how edge deletions can improve generalization, thus connecting spectral gap optimization to a seemingly disconnected objective of reducing computational resources by pruning graphs for lottery tickets. To this end, we propose a more effective spectral gap optimization framework to add or delete edges and demonstrate its effectiveness on large heterophilic datasets.