Learning Graph Neural Networks with Approximate Gradient Descent

TL;DR

This paper introduces a provably efficient algorithm for training one-hidden-layer graph neural networks, providing convergence guarantees and analyzing sample complexity, feature impact, and structure effects with empirical validation.

Contribution

It presents the first provably efficient training algorithm for GNNs with convergence guarantees and comprehensive theoretical analysis of sample complexity and structural factors.

Findings

Algorithm guarantees linear convergence to true parameters.

Sample complexity depends on nodes or graphs.

Feature dimension and GNN structure affect convergence rate.

Abstract

The first provably efficient algorithm for learning graph neural networks (GNNs) with one hidden layer for node information convolution is provided in this paper. Two types of GNNs are investigated, depending on whether labels are attached to nodes or graphs. A comprehensive framework for designing and analyzing convergence of GNN training algorithms is developed. The algorithm proposed is applicable to a wide range of activation functions including ReLU, Leaky ReLU, Sigmod, Softplus and Swish. It is shown that the proposed algorithm guarantees a linear convergence rate to the underlying true parameters of GNNs. For both types of GNNs, sample complexity in terms of the number of nodes or the number of graphs is characterized. The impact of feature dimension and GNN structure on the convergence rate is also theoretically characterized. Numerical experiments are further provided to validate our theoretical analysis.