Re-Think and Re-Design Graph Neural Networks in Spaces of Continuous Graph Diffusion Functionals

TL;DR

This paper introduces a continuous diffusion framework for GNNs, addressing over-smoothing and long-range dependencies by integrating variational analysis, total variation regularization, and neural transport equations, achieving state-of-the-art results.

Contribution

It proposes a novel continuous domain perspective for GNNs, incorporating variational analysis, TV regularization, and a GAN-based flow prediction model, advancing theoretical understanding and practical performance.

Findings

Achieves SOTA on Cora, Citeseer, Pubmed benchmarks.

Identifies over-smoothing as l2-norm graph gradient integral.

Introduces TV regularization to preserve community structures.

Abstract

Graph neural networks (GNNs) are widely used in domains like social networks and biological systems. However, the locality assumption of GNNs, which limits information exchange to neighboring nodes, hampers their ability to capture long-range dependencies and global patterns in graphs. To address this, we propose a new inductive bias based on variational analysis, drawing inspiration from the Brachistochrone problem. Our framework establishes a mapping between discrete GNN models and continuous diffusion functionals. This enables the design of application-specific objective functions in the continuous domain and the construction of discrete deep models with mathematical guarantees. To tackle over-smoothing in GNNs, we analyze the existing layer-by-layer graph embedding models and identify that they are equivalent to l2-norm integral functionals of graph gradients, which cause over-smoothing. Similar to edge-preserving filters in image denoising, we introduce total variation (TV) to align the graph diffusion pattern with global community topologies. Additionally, we devise a selective mechanism to address the trade-off between model depth and over-smoothing, which can be easily integrated into existing GNNs. Furthermore, we propose a novel generative adversarial network (GAN) that predicts spreading flows in graphs through a neural transport equation. To mitigate vanishing flows, we customize the objective function to minimize transportation within each community while maximizing inter-community flows. Our GNN models achieve state-of-the-art (SOTA) performance on popular graph learning benchmarks such as Cora, Citeseer, and Pubmed.