Graph Neural Network Training Systems: A Performance Comparison of Full-Graph and Mini-Batch

TL;DR

This paper empirically compares full-graph and mini-batch training systems for GNNs, revealing that mini-batch methods generally converge faster and achieve comparable or higher accuracy, emphasizing the need for cross-method evaluation.

Contribution

It provides the first comprehensive empirical comparison between full-graph and mini-batch GNN training systems, highlighting differences in convergence speed and accuracy.

Findings

Mini-batch systems converge faster than full-graph systems.

Mini-batch methods often reach equal or higher accuracy.

Cross-method comparisons using time-to-accuracy are crucial.

Abstract

Graph Neural Networks (GNNs) have gained significant attention in recent years due to their ability to learn representations of graph-structured data. Two common methods for training GNNs are mini-batch training and full-graph training. Since these two methods require different training pipelines and systems optimizations, two separate classes of GNN training systems emerged, each tailored for one method. Works that introduce systems belonging to a particular category predominantly compare them with other systems within the same category, offering limited or no comparison with systems from the other category. Some prior work also justifies its focus on one specific training method by arguing that it achieves higher accuracy than the alternative. The literature, however, has incomplete and contradictory evidence in this regard. In this paper, we provide a comprehensive empirical comparison of representative full-graph and mini-batch GNN training systems. We find that the mini-batch training systems consistently converge faster than the full-graph training ones across multiple datasets, GNN models, and system configurations. We also find that mini-batch training techniques converge to similar to or often higher accuracy values than full-graph training ones, showing that mini-batch sampling is not necessarily detrimental to accuracy. Our work highlights the importance of comparing systems across different classes, using time-to-accuracy rather than epoch time for performance comparison, and selecting appropriate hyperparameters for each training method separately.