Understanding and Improving Deep Graph Neural Networks: A Probabilistic Graphical Model Perspective

TL;DR

This paper introduces a probabilistic graphical model perspective to understand deep GNNs, unifies existing models under a common framework, and proposes CoGNet, a new GNN that outperforms state-of-the-art methods.

Contribution

It establishes a theoretical framework connecting deep GNNs via variational inference on probabilistic graphical models and designs a novel, more powerful GNN called CoGNet.

Findings

CoGNet outperforms SOTA models on citation and NLP tasks.

Deep GNNs can be interpreted as approximations of a fixed point equation.

The framework unifies various GNN architectures under a probabilistic inference perspective.

Abstract

Recently, graph-based models designed for downstream tasks have significantly advanced research on graph neural networks (GNNs). GNN baselines based on neural message-passing mechanisms such as GCN and GAT perform worse as the network deepens. Therefore, numerous GNN variants have been proposed to tackle this performance degradation problem, including many deep GNNs. However, a unified framework is still lacking to connect these existing models and interpret their effectiveness at a high level. In this work, we focus on deep GNNs and propose a novel view for understanding them. We establish a theoretical framework via inference on a probabilistic graphical model. Given the fixed point equation (FPE) derived from the variational inference on the Markov random fields, the deep GNNs, including JKNet, GCNII, DGCN, and the classical GNNs, such as GCN, GAT, and APPNP, can be regarded as different approximations of the FPE. Moreover, given this framework, more accurate approximations of FPE are brought, guiding us to design a more powerful GNN: coupling graph neural network (CoGNet). Extensive experiments are carried out on citation networks and natural language processing downstream tasks. The results demonstrate that the CoGNet outperforms the SOTA models.