Covered Forest: Fine-grained generalization analysis of graph neural networks

TL;DR

This paper investigates the generalization capabilities of message-passing graph neural networks (MPNNs), analyzing how graph structure, aggregation functions, and loss functions influence their ability to make accurate predictions beyond training data.

Contribution

It extends graph similarity theory to evaluate factors affecting MPNNs' generalization, providing both theoretical insights and empirical validation.

Findings

Graph structure significantly impacts MPNNs' generalization.

Aggregation functions influence the predictive power of MPNNs.

Loss functions shape the learning and generalization behavior.

Abstract

The expressive power of message-passing graph neural networks (MPNNs) is reasonably well understood, primarily through combinatorial techniques from graph isomorphism testing. However, MPNNs' generalization abilities -- making meaningful predictions beyond the training set -- remain less explored. Current generalization analyses often overlook graph structure, limit the focus to specific aggregation functions, and assume the impractical, hard-to-optimize $0$-$1$ loss function. Here, we extend recent advances in graph similarity theory to assess the influence of graph structure, aggregation, and loss functions on MPNNs' generalization abilities. Our empirical study supports our theoretical insights, improving our understanding of MPNNs' generalization properties.