GRACE: Gradient-aligned Reasoning Data Curation for Efficient Post-training

TL;DR

GRACE is a novel data curation method that scores individual reasoning steps based on gradient alignment and trajectory consistency, enabling efficient subset selection for training language models.

Contribution

It introduces a scalable, step-level scoring technique using internal model signals without external annotations, improving data efficiency in reasoning tasks.

Findings

GRACE achieves 108.8% of full-data performance with only 20% of the data.

It retains 100.2% performance with just 5% of the data.

Subsets selected by GRACE transfer effectively across different model backbones.

Abstract

Existing reasoning data curation pipelines score whole samples, treating every intermediate step as equally valuable. In reality, steps within a trace contribute very unevenly, and selecting reasoning data well requires assessing them individually. We present GRACE, a gradient-aligned curation method that views each reasoning trace as a sequence of optimization events and scores every step by two complementary signals: its alignment with the answer-oriented gradient direction, and its consistency with the preceding reasoning trajectory. Step-level scores are aggregated into a sample-level value for subset selection, using only the model's internal optimization signals and no external reward models or step annotations. To make this scalable, GRACE introduces a representation-level gradient proxy that estimates step-level alignment from token-level upstream signals in a single forward pass. Post-training Qwen3-VL-2B-Instruct on MMathCoT-1M, GRACE reaches 108.8% of the full-data performance with 20% of the data and retains 100.2% with only 5%, with subsets that transfer effectively across model backbones.