CondenseGraph: Communication-Efficient Distributed GNN Training via On-the-Fly Graph Condensation

TL;DR

CondenseGraph introduces a dynamic graph condensation method with error feedback to significantly reduce communication overhead in distributed GNN training, maintaining accuracy while improving scalability.

Contribution

It presents a novel on-the-fly graph condensation framework with error feedback, enabling adaptive compression during distributed GNN training.

Findings

Reduces communication volume by 40-60%.

Maintains comparable accuracy to full-precision training.

Speeds up training time significantly.

Abstract

Distributed Graph Neural Network (GNN) training suffers from substantial communication overhead due to the inherent neighborhood dependency in graph-structured data. This neighbor explosion problem requires workers to frequently exchange boundary node features across partitions, creating a communication bottleneck that severely limits training scalability. Existing approaches rely on static graph partitioning strategies that cannot adapt to dynamic network conditions. In this paper, we propose CondenseGraph, a novel communication-efficient framework for distributed GNN training. Our key innovation is an on-the-fly graph condensation mechanism that dynamically compresses boundary node features into compact super nodes before transmission. To compensate for the information loss introduced by compression, we develop a gradient-based error feedback mechanism that maintains convergence guarantees while reducing communication volume by 40-60%. Extensive experiments on four benchmark datasets demonstrate that CondenseGraph achieves comparable accuracy to full-precision baselines while significantly reducing communication costs and training time.