SPaCe: Unlocking Sample-Efficient Large Language Models Training With Self-Pace Curriculum Learning

TL;DR

SPaCe is a self-paced curriculum learning framework that improves large language model training efficiency by adaptively selecting data based on difficulty and model capability, reducing data needs significantly.

Contribution

The paper introduces SPaCe, a novel data reduction and adaptive sampling method that enhances training efficiency and performance of LLMs with fewer samples.

Findings

SPaCe achieves comparable or better accuracy than state-of-the-art methods.

It reduces training data requirements by up to 100 times.

Data clustering and adaptive selection are crucial for effectiveness.

Abstract

Large language models (LLMs) have shown strong reasoning capabilities when fine-tuned with reinforcement learning (RL). However, such methods require extensive data and compute, making them impractical under many realistic training budgets. Many existing pipelines sample training examples uniformly across steps or epochs, ignoring differences in difficulty, redundancy, and learning value, which slows learning and wastes computation. We propose \textbf{SPaCe}, a self-paced learning framework that enables efficient learning based on the capability of the model being trained through optimizing which data to use and when. First, we apply \emph{cluster-based data reduction} to partition training data by semantics and difficulty, extracting a compact yet diverse subset that reduces redundancy. Then, a \textit{multi-armed bandit} treats data clusters as arms, allocating training samples based on the model's solve rates and learning progress. Experiments across multiple reasoning benchmarks show that SPaCe achieves comparable or better accuracy than state-of-the-art baselines while using up to \(100\times\) fewer samples. Ablation studies and analyses further highlight the importance of both data clustering and adaptive selection. Our results demonstrate that carefully curated, performance-driven training curricula can unlock strong reasoning abilities in LLMs with minimal resources.