HopGNN: Boosting Distributed GNN Training Efficiency via Feature-Centric Model Migration

TL;DR

This paper introduces LeapGNN, a feature-centric framework for distributed GNN training that significantly reduces communication overhead and improves efficiency by bringing models to features, achieving up to 4.2x speedup.

Contribution

It proposes a novel feature-centric approach with micrograph-based training, feature pre-gathering, and merging strategies to enhance distributed GNN training efficiency.

Findings

Achieves up to 4.2x speedup over state-of-the-art methods.

Reduces remote feature retrieval and communication overhead.

Effectively minimizes kernel switches and synchronization costs.

Abstract

Distributed training of graph neural networks (GNNs) has become a crucial technique for processing large graphs. Prevalent GNN frameworks are model-centric, necessitating the transfer of massive graph vertex features to GNN models, which leads to a significant communication bottleneck. Recognizing that the model size is often significantly smaller than the feature size, we propose LeapGNN, a feature-centric framework that reverses this paradigm by bringing GNN models to vertex features. To make it truly effective, we first propose a micrograph-based training strategy that trains the model using a refined structure with superior locality to reduce remote feature retrieval. Then, we devise a feature pre-gathering approach that merges multiple fetch operations into a single one to eliminate redundant feature transmissions. Finally, we employ a micrograph-based merging method that adjusts the number of micrographs for each worker to minimize kernel switches and synchronization overhead. Our experimental results demonstrate that LeapGNN achieves a performance speedup of up to 4.2x compared to the state-of-the-art method, namely P3.