Residual connections provably mitigate oversmoothing in graph neural networks

TL;DR

This paper provides a theoretical analysis showing that residual connections in graph neural networks effectively prevent oversmoothing, a common problem that causes deep GNNs to lose expressive power, supported by numerical experiments.

Contribution

It offers the first explicit convergence rates for oversmoothing in deep GNNs and proves residual connections mitigate this issue across various parameter distributions.

Findings

Residual connections prevent oversmoothing in deep GNNs.

Explicit convergence rates for oversmoothing are derived.

Numerical experiments support the theoretical results.

Abstract

Graph neural networks (GNNs) have achieved remarkable empirical success in processing and representing graph-structured data across various domains. However, a significant challenge known as "oversmoothing" persists, where vertex features become nearly indistinguishable in deep GNNs, severely restricting their expressive power and practical utility. In this work, we analyze the asymptotic oversmoothing rates of deep GNNs with and without residual connections by deriving explicit convergence rates for a normalized vertex similarity measure. Our analytical framework is grounded in the multiplicative ergodic theorem. Furthermore, we demonstrate that adding residual connections effectively mitigates or prevents oversmoothing across several broad families of parameter distributions. The theoretical findings are strongly supported by numerical experiments.