GraphSculptor: Sculpting Pre-training Coreset for Graph Self-supervised Learning

TL;DR

GraphSculptor introduces a label-free coreset construction method for graph self-supervised learning, significantly reducing data and computational requirements while maintaining high downstream performance.

Contribution

It proposes a novel, unsupervised coreset construction approach combining structural and semantic diversity, with theoretical guarantees and practical efficiency.

Findings

A 10% coreset retains 99.6% of full-data performance.

Pre-training time is reduced by nearly 90%.

The method outperforms existing approaches in data efficiency.

Abstract

Graph self-supervised learning typically relies on large-scale unlabeled datasets, heavily inflating computational costs. However, empirical evidence suggests that these datasets contain substantial redundancy-our analysis reveals that uniformly subsampling 50% of graphs retains over 96% of downstream performance. To exploit this redundancy, we introduce GraphSculptor for pre-training coreset construction. Unlike methods dependent on additional training-time signals or limited solely to topological statistics, GraphSculptor provides a label-free solution that constructs coresets via two complementary perspectives: intrinsic structure and contextual semantics. Concretely, structural diversity is quantified using intrinsic graph statistics, yielding a structural feature vector for each graph, while semantic diversity is captured by utilizing a pre-trained language model to encode descriptions generated via graph-to-text. GraphSculptor integrates these signals into a unified metric space and performs cluster-aware selection to preserve joint structural-semantic diversity. We further derive a theoretical bound on the loss gap between coreset and full-data pre-training, offering theoretical motivation for our selection formulation. Extensive experiments demonstrate that GraphSculptor effectively sculpts the dataset: a 10% coreset achieves 99.6% of full-data performance while reducing pre-training time by nearly 90%, offering a scalable solution for data-efficient graph pre-training.