Convexified Message-Passing Graph Neural Networks

TL;DR

This paper introduces Convexified Message-Passing GNNs (CGNNs), a new framework that combines message-passing with convex optimization, leading to more efficient training, better theoretical understanding, and significantly improved accuracy on benchmark datasets.

Contribution

The paper proposes CGNNs, a convex optimization-based framework for GNNs, enabling efficient training, rigorous analysis, and superior performance over existing models.

Findings

CGNNs outperform leading GNN models by 10-40% accuracy.

Two-layer CGNNs have proven generalization guarantees.

Convex models can surpass non-convex models in accuracy and compactness.

Abstract

Graph Neural Networks (GNNs) are key tools for graph representation learning, demonstrating strong results across diverse prediction tasks. In this paper, we present Convexified Message-Passing Graph Neural Networks (CGNNs), a novel and general framework that combines the power of message-passing GNNs with the tractability of convex optimization. By mapping their nonlinear filters into a reproducing kernel Hilbert space, CGNNs transform training into a convex optimization problem, which projected gradient methods can solve both efficiently and optimally. Convexity further allows CGNNs' statistical properties to be analyzed accurately and rigorously. For two-layer CGNNs, we establish rigorous generalization guarantees, showing convergence to the performance of an optimal GNN. To scale to deeper architectures, we adopt a principled layer-wise training strategy. Experiments on benchmark datasets show that CGNNs significantly exceed the performance of leading GNN models, obtaining 10-40% higher accuracy in most cases, underscoring their promise as a powerful and principled method with strong theoretical foundations. In rare cases where improvements are not quantitatively substantial, the convex models either slightly exceed or match the baselines, stressing their robustness and wide applicability. Though over-parameterization is often used to enhance performance in non-convex models, we show that our CGNNs yield shallow convex models that can surpass non-convex ones in accuracy and model compactness.