ReInc: Scaling Training of Dynamic Graph Neural Networks

TL;DR

ReInc is a system that significantly improves the efficiency and scalability of training Dynamic Graph Neural Networks on large datasets by reusing computations, caching, and distributed strategies, achieving up to tenfold speedups.

Contribution

ReInc introduces novel caching and distributed training techniques specifically designed for DGNNs, enabling scalable training on large dynamic graphs.

Findings

Achieves up to 10x speedup over existing frameworks.

Supports various DGNN architectures and real-world datasets.

Reduces communication overhead in distributed training.

Abstract

Dynamic Graph Neural Networks (DGNNs) have gained widespread attention due to their applicability in diverse domains such as traffic network prediction, epidemiological forecasting, and social network analysis. In this paper, we present ReInc, a system designed to enable efficient and scalable training of DGNNs on large-scale graphs. ReInc introduces key innovations that capitalize on the unique combination of Graph Neural Networks (GNNs) and Recurrent Neural Networks (RNNs) inherent in DGNNs. By reusing intermediate results and incrementally computing aggregations across consecutive graph snapshots, ReInc significantly enhances computational efficiency. To support these optimizations, ReInc incorporates a novel two-level caching mechanism with a specialized caching policy aligned to the DGNN execution workflow. Additionally, ReInc addresses the challenges of managing structural and temporal dependencies in dynamic graphs through a new distributed training strategy. This approach eliminates communication overheads associated with accessing remote features and redistributing intermediate results. Experimental results demonstrate that ReInc achieves up to an order of magnitude speedup compared to state-of-the-art frameworks, tested across various dynamic GNN architectures and real-world graph datasets.