Convergence of Invariant Graph Networks

TL;DR

This paper studies the convergence properties of Invariant Graph Networks (IGNs) on graphs sampled from graphons, establishing conditions for convergence and demonstrating limitations and possibilities through theoretical proofs and experiments.

Contribution

It provides the first convergence analysis of IGNs on graphons, introduces the concept of IGN-small, and explores convergence under different sampling settings.

Findings

Linear layers of k-IGN are stable under new interpretations.

Convergence of k-IGN is proven when edge weights are accessible.

Convergence is not possible for standard IGNs with only 0-1 adjacency matrices.

Abstract

Although theoretical properties such as expressive power and over-smoothing of graph neural networks (GNN) have been extensively studied recently, its convergence property is a relatively new direction. In this paper, we investigate the convergence of one powerful GNN, Invariant Graph Network (IGN) over graphs sampled from graphons. We first prove the stability of linear layers for general $k$-IGN (of order $k$) based on a novel interpretation of linear equivariant layers. Building upon this result, we prove the convergence of $k$-IGN under the model of \citet{ruiz2020graphon}, where we access the edge weight but the convergence error is measured for graphon inputs. Under the more natural (and more challenging) setting of \citet{keriven2020convergence} where one can only access 0-1 adjacency matrix sampled according to edge probability, we first show a negative result that the convergence of any IGN is not possible. We then obtain the convergence of a subset of IGNs, denoted as IGN-small, after the edge probability estimation. We show that IGN-small still contains function class rich enough that can approximate spectral GNNs arbitrarily well. Lastly, we perform experiments on various graphon models to verify our statements.