Training Graph Neural Networks on Growing Stochastic Graphs

TL;DR

This paper introduces a method for training Graph Neural Networks directly on large, growing graphs by leveraging graphons, which ensures convergence to a near-optimal solution on the limiting graph structure.

Contribution

The paper proposes a novel training approach called learning by transference that grows the graph during training and converges to a neighborhood of a stationary point on the graphon.

Findings

Method converges to a neighborhood of a stationary point on the graphon.

Numerical experiments validate the effectiveness of the approach.

Abstract

Graph Neural Networks (GNNs) rely on graph convolutions to exploit meaningful patterns in networked data. Based on matrix multiplications, convolutions incur in high computational costs leading to scalability limitations in practice. To overcome these limitations, proposed methods rely on training GNNs in smaller number of nodes, and then transferring the GNN to larger graphs. Even though these methods are able to bound the difference between the output of the GNN with different number of nodes, they do not provide guarantees against the optimal GNN on the very large graph. In this paper, we propose to learn GNNs on very large graphs by leveraging the limit object of a sequence of growing graphs, the graphon. We propose to grow the size of the graph as we train, and we show that our proposed methodology -- learning by transference -- converges to a neighborhood of a first order stationary point on the graphon data. A numerical experiment validates our proposed approach.