DeGNN: Characterizing and Improving Graph Neural Networks with Graph Decomposition

TL;DR

This paper analyzes the oversmoothing problem in deep GCNs from an information-theoretic perspective and introduces DeGNN, a graph decomposition method that enhances GNN performance and mitigates oversmoothing.

Contribution

It characterizes oversmoothing in GCNs using mutual information and proposes DeGNN, an automatic graph decomposition algorithm, to improve GNN effectiveness and achieve state-of-the-art results.

Findings

DeGNN significantly boosts GNN performance on benchmark datasets.

Graph decomposition can weaken the exponential convergence of mutual information.

DeGNN achieves state-of-the-art results across multiple GNN architectures.

Abstract

Despite the wide application of Graph Convolutional Network (GCN), one major limitation is that it does not benefit from the increasing depth and suffers from the oversmoothing problem. In this work, we first characterize this phenomenon from the information-theoretic perspective and show that under certain conditions, the mutual information between the output after $l$ layers and the input of GCN converges to 0 exponentially with respect to $l$. We also show that, on the other hand, graph decomposition can potentially weaken the condition of such convergence rate, which enabled our analysis for GraphCNN. While different graph structures can only benefit from the corresponding decomposition, in practice, we propose an automatic connectivity-aware graph decomposition algorithm, DeGNN, to improve the performance of general graph neural networks. Extensive experiments on widely adopted benchmark datasets demonstrate that DeGNN can not only significantly boost the performance of corresponding GNNs, but also achieves the state-of-the-art performances.