Rethinking Graph Regularization for Graph Neural Networks

TL;DR

This paper critically examines the limited benefits of traditional graph Laplacian regularization in GNNs, introduces a new Propagation-regularization method, and demonstrates its effectiveness in enhancing GNN performance across various tasks.

Contribution

The paper proposes Propagation-regularization (P-reg), a novel variant that improves GNNs by infusing additional information and achieving capacity comparable to infinite-depth graph convolutional networks.

Findings

P-reg significantly boosts GNN performance on multiple datasets.

Traditional Laplacian regularization offers minimal benefits to GNNs.

P-reg captures information beyond standard regularization, enhancing both node and graph-level tasks.

Abstract

The graph Laplacian regularization term is usually used in semi-supervised representation learning to provide graph structure information for a model $f(X)$. However, with the recent popularity of graph neural networks (GNNs), directly encoding graph structure $A$ into a model, i.e., $f(A, X)$, has become the more common approach. While we show that graph Laplacian regularization brings little-to-no benefit to existing GNNs, and propose a simple but non-trivial variant of graph Laplacian regularization, called Propagation-regularization (P-reg), to boost the performance of existing GNN models. We provide formal analyses to show that P-reg not only infuses extra information (that is not captured by the traditional graph Laplacian regularization) into GNNs, but also has the capacity equivalent to an infinite-depth graph convolutional network. We demonstrate that P-reg can effectively boost the performance of existing GNN models on both node-level and graph-level tasks across many different datasets.