Demystifying Distributed Training of Graph Neural Networks for Link Prediction

TL;DR

This paper investigates the challenges of distributed training of graph neural networks for link prediction, identifies key issues, and proposes SpLPG, a sparsification method that reduces communication costs while maintaining accuracy.

Contribution

It introduces SpLPG, a novel graph sparsification approach that mitigates performance degradation in distributed GNN training for link prediction, reducing communication overhead significantly.

Findings

SpLPG reduces communication overhead by up to 80%.

Performance degradation is caused by information loss and negative sampling methods.

Sharing complete graph information preserves accuracy but is costly.

Abstract

Graph neural networks (GNNs) are powerful tools for solving graph-related problems. Distributed GNN frameworks and systems enhance the scalability of GNNs and accelerate model training, yet most are optimized for node classification. Their performance on link prediction remains underexplored. This paper demystifies distributed training of GNNs for link prediction by investigating the issue of performance degradation when each worker trains a GNN on its assigned partitioned subgraph without having access to the entire graph. We discover that the main sources of the issue come from not only the information loss caused by graph partitioning but also the ways of drawing negative samples during model training. While sharing the complete graph information with each worker resolves the issue and preserves link prediction accuracy, it incurs a high communication cost. We propose SpLPG, which effectively leverages graph sparsification to mitigate the issue of performance degradation at a reduced communication cost. Experiment results on several public real-world datasets demonstrate the effectiveness of SpLPG, which reduces the communication overhead by up to about 80% while mostly preserving link prediction accuracy.