Towards Deeper Graph Neural Networks

TL;DR

This paper systematically analyzes the over-smoothing problem in deep graph neural networks, decouples key operations to enable deeper models, and proposes DAGNN to adaptively leverage larger receptive fields, improving performance.

Contribution

It provides a theoretical and empirical analysis of over-smoothing, introduces a decoupled graph convolution approach, and proposes DAGNN for deeper graph neural networks.

Findings

Decoupling transformation and propagation improves deep GNN performance.

Deeper GNNs can learn from larger receptive fields with proper design.

DAGNN outperforms existing methods on multiple datasets.

Abstract

Graph neural networks have shown significant success in the field of graph representation learning. Graph convolutions perform neighborhood aggregation and represent one of the most important graph operations. Nevertheless, one layer of these neighborhood aggregation methods only consider immediate neighbors, and the performance decreases when going deeper to enable larger receptive fields. Several recent studies attribute this performance deterioration to the over-smoothing issue, which states that repeated propagation makes node representations of different classes indistinguishable. In this work, we study this observation systematically and develop new insights towards deeper graph neural networks. First, we provide a systematical analysis on this issue and argue that the key factor compromising the performance significantly is the entanglement of representation transformation and propagation in current graph convolution operations. After decoupling these two operations, deeper graph neural networks can be used to learn graph node representations from larger receptive fields. We further provide a theoretical analysis of the above observation when building very deep models, which can serve as a rigorous and gentle description of the over-smoothing issue. Based on our theoretical and empirical analysis, we propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields. A set of experiments on citation, co-authorship, and co-purchase datasets have confirmed our analysis and insights and demonstrated the superiority of our proposed methods.