Convergence of Message Passing Graph Neural Networks with Generic Aggregation On Large Random Graphs

TL;DR

This paper proves the convergence of a broad class of message passing graph neural networks on large random graphs to their continuous limits, extending previous results beyond mean-based aggregations.

Contribution

It extends convergence results to include various aggregation functions like attention, max, and moment-based methods, beyond normalized mean aggregations.

Findings

Non-asymptotic convergence bounds with high probability

Convergence proven for multiple aggregation functions including attention and moments

Separate analysis and convergence rate for max aggregation

Abstract

We study the convergence of message passing graph neural networks on random graph models to their continuous counterpart as the number of nodes tends to infinity. Until now, this convergence was only known for architectures with aggregation functions in the form of normalized means, or, equivalently, of an application of classical operators like the adjacency matrix or the graph Laplacian. We extend such results to a large class of aggregation functions, that encompasses all classically used message passing graph neural networks, such as attention-based message passing, max convolutional message passing, (degree-normalized) convolutional message passing, or moment-based aggregation message passing. Under mild assumptions, we give non-asymptotic bounds with high probability to quantify this convergence. Our main result is based on the McDiarmid inequality. Interestingly, this result does not apply to the case where the aggregation is a coordinate-wise maximum. We treat this case separately and obtain a different convergence rate.