Transductive Generalization via Optimal Transport and Its Application to Graph Node Classification

TL;DR

This paper introduces new transductive generalization bounds for graph neural networks using optimal transport, which are computationally feasible and better aligned with empirical results, especially in graph node classification.

Contribution

It develops representation-based bounds in a transductive setting using Wasserstein distances, revealing depth-related effects on GNN generalization.

Findings

Bounds are efficiently computable and correlate with empirical performance.

GNN aggregation affects representation distributions, balancing intra-class and inter-class separation.

Depth influences generalization error non-monotonically, as captured by the bounds.

Abstract

Many existing transductive bounds rely on classical complexity measures that are computationally intractable and often misaligned with empirical behavior. In this work, we establish new representation-based generalization bounds in a distribution-free transductive setting, where learned representations are dependent, and test features are accessible during training. We derive global and class-wise bounds via optimal transport, expressed in terms of Wasserstein distances between encoded feature distributions. We demonstrate that our bounds are efficiently computable and strongly correlate with empirical generalization in graph node classification, improving upon classical complexity measures. Additionally, our analysis reveals how the GNN aggregation process transforms the representation distributions, inducing a trade-off between intra-class concentration and inter-class separation. This yields depth-dependent characterizations that capture the non-monotonic relationship between depth and generalization error observed in practice. The code is available at https://github.com/ml-postech/Transductive-OT-Gen-Bound.