DGC: Training Dynamic Graphs with Spatio-Temporal Non-Uniformity using Graph Partitioning by Chunks

TL;DR

This paper introduces DGC, a distributed training system for dynamic graph neural networks that uses a novel chunk-based graph partitioning method to handle non-uniform spatio-temporal structures, significantly accelerating training.

Contribution

DGC proposes a fast graph coarsening-based partitioning method and efficient runtime techniques for scalable DGNN training on non-uniform dynamic graphs.

Findings

Achieves 1.25x to 7.52x speedup over state-of-the-art methods.

Effective on 3 DGNN models and 4 dynamic graph datasets.

Handles non-uniform graph structures with improved efficiency.

Abstract

Dynamic Graph Neural Network (DGNN) has shown a strong capability of learning dynamic graphs by exploiting both spatial and temporal features. Although DGNN has recently received considerable attention by AI community and various DGNN models have been proposed, building a distributed system for efficient DGNN training is still challenging. It has been well recognized that how to partition the dynamic graph and assign workloads to multiple GPUs plays a critical role in training acceleration. Existing works partition a dynamic graph into snapshots or temporal sequences, which only work well when the graph has uniform spatio-temporal structures. However, dynamic graphs in practice are not uniformly structured, with some snapshots being very dense while others are sparse. To address this issue, we propose DGC, a distributed DGNN training system that achieves a 1.25x - 7.52x speedup over the state-of-the-art in our testbed. DGC's success stems from a new graph partitioning method that partitions dynamic graphs into chunks, which are essentially subgraphs with modest training workloads and few inter connections. This partitioning algorithm is based on graph coarsening, which can run very fast on large graphs. In addition, DGC has a highly efficient run-time, powered by the proposed chunk fusion and adaptive stale aggregation techniques. Extensive experimental results on 3 typical DGNN models and 4 popular dynamic graph datasets are presented to show the effectiveness of DGC.