Rethinking Generalization in Graph Neural Networks: A Structural Complexity Perspective

TL;DR

This paper investigates how graph structure influences GNN generalization, revealing that more edges can cause overfitting, and proposes a regularization method to improve generalization by controlling structural complexity.

Contribution

It provides a theoretical analysis linking structural complexity to GNN generalization and introduces a regularization technique based on structural entropy.

Findings

Adding more edges can lead to overfitting in GNNs.

Structural complexity explicitly affects GNN generalization bounds.

Regularizing structural entropy improves GNN generalization performance.

Abstract

Graph neural networks (GNNs) have emerged as a fundamental tool for learning from graph-structured data, achieving strong performance across a wide range of applications. However, understanding their generalization capabilities remains challenging due to the complex structural dependencies inherent in such data. Existing generalization analyses largely follow the classical machine learning paradigm, focusing primarily on model complexity while overlooking the fundamental role of graph structure. Therefore, in this work, we systematically investigate this role by asking: does the graph structure actually influence generalization, and if so, by how much? To answer the first question and validate our intuition, we theoretically prove that incorporating more edges into the prediction process transforms the input representations to be overly accommodating to the output model, thereby inducing overfitting. To address the second question, we formulate a structural complexity measure based on the number of effective edges and derive a Rademacher complexity-based generalization bound. In doing so, we demonstrate that GNN generalization depends explicitly on structural complexity, alongside traditional parameter-dependent factors. Motivated by these theoretical findings, we propose a structural entropy regularization method. This approach controls structural complexity by regulating effective edges to balance underfitting and overfitting, ultimately improving the generalization performance of GNNs.