Generalization of Geometric Graph Neural Networks with Lipschitz Loss Functions

TL;DR

This paper analyzes the generalization ability of geometric GNNs on sampled manifolds, showing that training on one large graph can generalize to unseen graphs, supported by theoretical proofs and experiments.

Contribution

It provides the first theoretical analysis of GNN generalization over sampled manifolds, demonstrating that a single large graph suffices for generalization.

Findings

Generalization gap decreases with more sample points

Training on one large graph can generalize to unseen graphs

Experimental validation on real-world datasets

Abstract

In this paper, we study the generalization capabilities of geometric graph neural networks (GNNs). We consider GNNs over a geometric graph constructed from a finite set of randomly sampled points over an embedded manifold with topological information captured. We prove a generalization gap between the optimal empirical risk and the optimal statistical risk of this GNN, which decreases with the number of sampled points from the manifold and increases with the dimension of the underlying manifold. This generalization gap ensures that the GNN trained on a graph on a set of sampled points can be utilized to process other unseen graphs constructed from the same underlying manifold. The most important observation is that the generalization capability can be realized with one large graph instead of being limited to the size of the graph as in previous results. The generalization gap is derived based on the non-asymptotic convergence result of a GNN on the sampled graph to the underlying manifold neural networks (MNNs). We verify this theoretical result with experiments on multiple real-world datasets.