Reducing Memory Contention and I/O Congestion for Disk-based GNN Training

TL;DR

GNNDrive is a novel system that reduces memory contention and I/O congestion in disk-based GNN training, significantly improving training speed on large graphs by optimizing buffer management and asynchronous data loading.

Contribution

It introduces GNNDrive, which minimizes memory use and I/O bottlenecks, enabling faster training of GNNs on large-scale graphs compared to existing systems.

Findings

GNNDrive outperforms state-of-the-art systems by up to 16.9x in training speed.

It effectively reduces memory contention and I/O congestion.

Experimental results demonstrate significant performance improvements.

Abstract

Graph neural networks (GNNs) gain wide popularity. Large graphs with high-dimensional features become common and training GNNs on them is non-trivial on an ordinary machine. Given a gigantic graph, even sample-based GNN training cannot work efficiently, since it is difficult to keep the graph's entire data in memory during the training process. Leveraging a solid-state drive (SSD) or other storage devices to extend the memory space has been studied in training GNNs. Memory and I/Os are hence critical for effectual disk-based training. We find that state-of-the-art (SoTA) disk-based GNN training systems severely suffer from issues like the memory contention between a graph's topological and feature data, and severe I/O congestion upon loading data from SSD for training. We accordingly develop GNNDrive. GNNDrive 1) minimizes the memory footprint with holistic buffer management across sampling and extracting, and 2) avoids I/O congestion through a strategy of asynchronous feature extraction. It also avoids costly data preparation on the critical path and makes the most of software and hardware resources. Experiments show that GNNDrive achieves superior performance. For example, when training with the Papers100M dataset and GraphSAGE model, GNNDrive is faster than SoTA PyG+, Ginex, and MariusGNN by 16.9x, 2.6x, and 2.7x, respectively.