Bonsai: Gradient-free Graph Condensation for Node Classification

TL;DR

Bonsai introduces a gradient-free, linear-time graph condensation method that efficiently compresses datasets for node classification, outperforming existing methods in speed and accuracy while being model-agnostic and robust.

Contribution

Bonsai is the first linear-time, model-agnostic graph condensation algorithm based on computation trees, addressing key limitations of prior gradient-based methods.

Findings

Outperforms existing baselines in accuracy across 7 datasets.

Operates 22 times faster on average than previous methods.

Provides mathematical guarantees on approximation strategies.

Abstract

Graph condensation has emerged as a promising avenue to enable scalable training of GNNs by compressing the training dataset while preserving essential graph characteristics. Our study uncovers significant shortcomings in current graph condensation techniques. First, the majority of the algorithms paradoxically require training on the full dataset to perform condensation. Second, due to their gradient-emulating approach, these methods require fresh condensation for any change in hyperparameters or GNN architecture, limiting their flexibility and reusability. Finally, they fail to achieve substantial size reduction due to synthesizing fully-connected, edge-weighted graphs. To address these challenges, we present Bonsai, a novel graph condensation method empowered by the observation that \textit{computation trees} form the fundamental processing units of message-passing GNNs. Bonsai condenses datasets by encoding a careful selection of \textit{exemplar} trees that maximize the representation of all computation trees in the training set. This unique approach imparts Bonsai as the first linear-time, model-agnostic graph condensation algorithm for node classification that outperforms existing baselines across $7$ real-world datasets on accuracy, while being $22$ times faster on average. Bonsai is grounded in rigorous mathematical guarantees on the adopted approximation strategies making it robust to GNN architectures, datasets, and parameters.