BatchGNN: Efficient CPU-Based Distributed GNN Training on Very Large Graphs

TL;DR

BatchGNN is a distributed CPU system that significantly accelerates training on very large graphs by reducing communication and sampling overhead through innovative batching and caching techniques.

Contribution

It introduces macrobatching, integrated graph partitioning, and feature caching to improve CPU-based GNN training efficiency on terabyte-scale graphs.

Findings

Achieves 3x speedup over DistDGL on multiple GNN models

Outperforms GPU-based systems P^3 and DistDGLv2 in runtime

Scales effectively to terabyte-sized graphs

Abstract

We present BatchGNN, a distributed CPU system that showcases techniques that can be used to efficiently train GNNs on terabyte-sized graphs. It reduces communication overhead with macrobatching in which multiple minibatches' subgraph sampling and feature fetching are batched into one communication relay to reduce redundant feature fetches when input features are static. BatchGNN provides integrated graph partitioning and native GNN layer implementations to improve runtime, and it can cache aggregated input features to further reduce sampling overhead. BatchGNN achieves an average $3\times$ speedup over DistDGL on three GNN models trained on OGBN graphs, outperforms the runtimes reported by distributed GPU systems $P^3$ and DistDGLv2, and scales to a terabyte-sized graph.