SpanGNN: Towards Memory-Efficient Graph Neural Networks via Spanning Subgraph Training

TL;DR

SpanGNN introduces a memory-efficient training method for GNNs using spanning subgraphs and edge sampling strategies, reducing peak memory usage while maintaining high model accuracy on large graphs.

Contribution

The paper proposes SpanGNN, a novel GNN training approach that constructs spanning subgraphs and employs edge sampling to improve memory efficiency without sacrificing accuracy.

Findings

SpanGNN significantly reduces peak memory usage compared to full-graph training.

SpanGNN maintains comparable or better accuracy than mini-batch GNN training.

Experimental results demonstrate the effectiveness of spanning subgraphs and edge sampling strategies.

Abstract

Graph Neural Networks (GNNs) have superior capability in learning graph data. Full-graph GNN training generally has high accuracy, however, it suffers from large peak memory usage and encounters the Out-of-Memory problem when handling large graphs. To address this memory problem, a popular solution is mini-batch GNN training. However, mini-batch GNN training increases the training variance and sacrifices the model accuracy. In this paper, we propose a new memory-efficient GNN training method using spanning subgraph, called SpanGNN. SpanGNN trains GNN models over a sequence of spanning subgraphs, which are constructed from empty structure. To overcome the excessive peak memory consumption problem, SpanGNN selects a set of edges from the original graph to incrementally update the spanning subgraph between every epoch. To ensure the model accuracy, we introduce two types of edge sampling strategies (i.e., variance-reduced and noise-reduced), and help SpanGNN select high-quality edges for the GNN learning. We conduct experiments with SpanGNN on widely used datasets, demonstrating SpanGNN's advantages in the model performance and low peak memory usage.