Not too little, not too much: a theoretical analysis of graph (over)smoothing

TL;DR

This paper provides a rigorous analysis of graph smoothing in linear GNNs, demonstrating how finite mean aggregation can improve learning by balancing information preservation and oversmoothing.

Contribution

It offers the first analysis showing both beneficial finite smoothing and oversmoothing in linear GNNs, with theoretical insights into how mean aggregation affects data directions and community structures.

Findings

Finite mean aggregation improves regression and classification performance.

Graph smoothing shrinks non-principal data directions faster, aiding learning.

Smoothing within communities enhances classification before oversmoothing occurs.

Abstract

We analyze graph smoothing with \emph{mean aggregation}, where each node successively receives the average of the features of its neighbors. Indeed, it has quickly been observed that Graph Neural Networks (GNNs), which generally follow some variant of Message-Passing (MP) with repeated aggregation, may be subject to the oversmoothing phenomenon: by performing too many rounds of MP, the node features tend to converge to a non-informative limit. In the case of mean aggregation, for connected graphs, the node features become constant across the whole graph. At the other end of the spectrum, it is intuitively obvious that some MP rounds are necessary, but existing analyses do not exhibit both phenomena at once: beneficial ``finite'' smoothing and oversmoothing in the limit. In this paper, we consider simplified linear GNNs, and rigorously analyze two examples for which a finite number of mean aggregation steps provably improves the learning performance, before oversmoothing kicks in. We consider a latent space random graph model, where node features are partial observations of the latent variables and the graph contains pairwise relationships between them. We show that graph smoothing restores some of the lost information, up to a certain point, by two phenomenon: graph smoothing shrinks non-principal directions in the data faster than principal ones, which is useful for regression, and shrinks nodes within communities faster than they collapse together, which improves classification.