A Survey on Oversmoothing in Graph Neural Networks

TL;DR

This survey comprehensively reviews the phenomenon of over-smoothing in graph neural networks, introduces new quantitative measures, and evaluates mitigation strategies across various graph datasets, highlighting challenges in deep GNN design.

Contribution

It unifies existing over-smoothing definitions, proposes new measures, and empirically assesses mitigation methods, extending the concept to continuous-time GNNs.

Findings

Over-smoothing leads to exponential similarity convergence in node features.

Mitigation strategies improve GNN depth but do not guarantee expressiveness.

Over-smoothing is a necessary but not sufficient condition for deep GNN effectiveness.

Abstract

Node features of graph neural networks (GNNs) tend to become more similar with the increase of the network depth. This effect is known as over-smoothing, which we axiomatically define as the exponential convergence of suitable similarity measures on the node features. Our definition unifies previous approaches and gives rise to new quantitative measures of over-smoothing. Moreover, we empirically demonstrate this behavior for several over-smoothing measures on different graphs (small-, medium-, and large-scale). We also review several approaches for mitigating over-smoothing and empirically test their effectiveness on real-world graph datasets. Through illustrative examples, we demonstrate that mitigating over-smoothing is a necessary but not sufficient condition for building deep GNNs that are expressive on a wide range of graph learning tasks. Finally, we extend our definition of over-smoothing to the rapidly emerging field of continuous-time GNNs.