Generalization, Expressivity, and Universality of Graph Neural Networks on Attributed Graphs
Levi Rauchwerger, Stefanie Jegelka, Ron Levie
TL;DR
This paper establishes the theoretical foundations for the universality and generalization capabilities of graph neural networks on attributed graphs, introducing new metrics and proving key properties like Lipschitz continuity and compactness.
Contribution
It introduces hierarchical optimal transport-based pseudometrics for attributed graphs and proves a universal approximation theorem and generalization bounds for GNNs.
Findings
GNNs are Lipschitz continuous with respect to the proposed pseudometrics.
The space of all attributed graphs is relatively compact under these metrics.
A universal approximation theorem for GNNs on attributed graphs is established.
Abstract
We analyze the universality and generalization of graph neural networks (GNNs) on attributed graphs, i.e., with node attributes. To this end, we propose pseudometrics over the space of all attributed graphs that describe the fine-grained expressivity of GNNs. Namely, GNNs are both Lipschitz continuous with respect to our pseudometrics and can separate attributed graphs that are distant in the metric. Moreover, we prove that the space of all attributed graphs is relatively compact with respect to our metrics. Based on these properties, we prove a universal approximation theorem for GNNs and generalization bounds for GNNs on any data distribution of attributed graphs. The proposed metrics compute the similarity between the structures of attributed graphs via a hierarchical optimal transport between computation trees. Our work extends and unites previous approaches which either derived…
Peer Reviews
Decision·ICLR 2025 Poster
* The proposed a noval pseudometric for graphs based on iterated degree measures and optimal transport. * The authors show MPNNs are universal and Lipschitz under this pseudometric. * The authors show a bidirectional connections between the metric and MPNN output. * A generalisation theorem is developed based the metric. * The framework unifies MPNNs on graph and graphon.
* The proposed metric only applies to MPNN/1-WL GNNs, and cannot be used to analyse more powerful GNNs. * Related works are insufficiently discussed. In particualr, the proposed metric seems realted to [1][2], but this connection has not been discussed. * The metric is expensive to compute, while its complexity is linear to $L$, it is in the order of $n^5$ which hinders its practicality * The MPNN analysis limits to normalized sum aggregation, which is less commonly used. * There are too many s
1. The theoretical analysis is solid and thorough. 2. This work enhances the understanding of GNNs. The work introduces a new framework that unifies generalization and expressivity theories for attributed graphs, a notable achievement in GNN research. 3. The idea of introducing OT to this problem is quite novel and fancy.
See the questions below.
1. The theoretical analysis is solid and strong. Especially, this work tries to provide an analysis that is as universal as possible with the fewest assumptions. 2. The discussion of MPNNs and tree mover's distance in Section 4 is interesting and novel to me.
1. The presentation is a bit strange. The main results in Section 5 appear too late on almost page 9 in the main body of the paper. Moreover, the universal approximation and the generalization bound seem independent of the previous analysis of graph/graphon distances. It is better to move Section 5 earlier and elaborate more about the connections with previous sections. 2. Section 5 is not insightful enough. For example, although Theorem 7 makes contributions by requiring fewer assumptions, ex
Videos
Taxonomy
TopicsNeural Networks and Applications · Graph Theory and Algorithms · Advanced Graph Neural Networks