Addressing Over-Smoothing in Graph Neural Networks via Deep Supervision

TL;DR

This paper introduces Deeply-Supervised GNNs (DSGNNs), which incorporate supervision at all layers to combat over-smoothing, leading to improved performance on node and graph prediction tasks.

Contribution

The paper proposes DSGNNs, a novel deep supervision approach that effectively mitigates over-smoothing in deep GNNs, enhancing their predictive capabilities.

Findings

DSGNNs resist over-smoothing in deep architectures.

DSGNNs outperform benchmarks on node prediction tasks.

DSGNNs improve graph property prediction accuracy.

Abstract

Learning useful node and graph representations with graph neural networks (GNNs) is a challenging task. It is known that deep GNNs suffer from over-smoothing where, as the number of layers increases, node representations become nearly indistinguishable and model performance on the downstream task degrades significantly. To address this problem, we propose deeply-supervised GNNs (DSGNNs), i.e., GNNs enhanced with deep supervision where representations learned at all layers are used for training. We show empirically that DSGNNs are resilient to over-smoothing and can outperform competitive benchmarks on node and graph property prediction problems.