Generalization and Representational Limits of Graph Neural Networks

TL;DR

This paper investigates the fundamental limitations of graph neural networks (GNNs) in computing certain graph properties and provides new, tighter generalization bounds that account for their local permutation invariance.

Contribution

It proves that standard and spatial GNNs cannot compute some important graph properties relying solely on local information, and introduces the first data-dependent generalization bounds for message passing GNNs.

Findings

Certain graph properties are uncomputable by local GNNs.

New formalism for analyzing GNN capabilities.

Tighter generalization bounds than previous VC-dimension estimates.

Abstract

We address two fundamental questions about graph neural networks (GNNs). First, we prove that several important graph properties cannot be computed by GNNs that rely entirely on local information. Such GNNs include the standard message passing models, and more powerful spatial variants that exploit local graph structure (e.g., via relative orientation of messages, or local port ordering) to distinguish neighbors of each node. Our treatment includes a novel graph-theoretic formalism. Second, we provide the first data dependent generalization bounds for message passing GNNs. This analysis explicitly accounts for the local permutation invariance of GNNs. Our bounds are much tighter than existing VC-dimension based guarantees for GNNs, and are comparable to Rademacher bounds for recurrent neural networks.