Rethinking and Accelerating Graph Condensation: A Training-Free Approach with Class Partition

TL;DR

This paper introduces a training-free graph condensation method called CGC that uses class partitioning and clustering to efficiently create small, informative graphs for GNN training, significantly reducing computation time and improving accuracy.

Contribution

The paper proposes a novel training-free graph condensation framework that transforms the optimization into a class partition problem, eliminating iterative training and achieving faster, more accurate graph condensation.

Findings

CGC condenses large graphs within 30 seconds.

CGC achieves 100-10,000x speedup over state-of-the-art methods.

CGC improves condensation accuracy by up to 4.2%.

Abstract

The increasing prevalence of large-scale graphs poses a significant challenge for graph neural network training, attributed to their substantial computational requirements. In response, graph condensation (GC) emerges as a promising data-centric solution aiming to substitute the large graph with a small yet informative condensed graph to facilitate data-efficient GNN training. However, existing GC methods suffer from intricate optimization processes, necessitating excessive computing resources and training time. In this paper, we revisit existing GC optimization strategies and identify two pervasive issues therein: (1) various GC optimization strategies converge to coarse-grained class-level node feature matching between the original and condensed graphs; (2) existing GC methods rely on a Siamese graph network architecture that requires time-consuming bi-level optimization with iterative gradient computations. To overcome these issues, we propose a training-free GC framework termed Class-partitioned Graph Condensation (CGC), which refines the node distribution matching from the class-to-class paradigm into a novel class-to-node paradigm, transforming the GC optimization into a class partition problem which can be efficiently solved by any clustering methods. Moreover, CGC incorporates a pre-defined graph structure to enable a closed-form solution for condensed node features, eliminating the need for back-and-forth gradient descent in existing GC approaches. Extensive experiments demonstrate that CGC achieves an exceedingly efficient condensation process with advanced accuracy. Compared with the state-of-the-art GC methods, CGC condenses the Ogbn-products graph within 30 seconds, achieving a speedup ranging from $10^2$X to $10^4$X and increasing accuracy by up to 4.2%.