SCOPE: Compress Mathematical Reasoning Steps for Efficient Automated Process Annotation

TL;DR

SCOPE introduces a compression-based method for annotating mathematical reasoning steps that significantly reduces computational costs and improves performance of Process Reward Models.

Contribution

The paper presents SCOPE, a novel compression approach that constructs a prefix tree from reasoning steps, drastically reducing annotation costs and resource usage.

Findings

Constructed a dataset with 196K samples using 95% less resources.

PRMs trained on SCOPE outperform existing annotation methods.

Achieved better results on Best-of-N and ProcessBench benchmarks.

Abstract

Process Reward Models (PRMs) have demonstrated promising results in mathematical reasoning, but existing process annotation approaches, whether through human annotations or Monte Carlo simulations, remain computationally expensive. In this paper, we introduce Step COmpression for Process Estimation (SCOPE), a novel compression-based approach that significantly reduces annotation costs. We first translate natural language reasoning steps into code and normalize them through Abstract Syntax Tree, then merge equivalent steps to construct a prefix tree. Unlike simulation-based methods that waste numerous samples on estimation, SCOPE leverages a compression-based prefix tree where each root-to-leaf path serves as a training sample, reducing the complexity from $O(NMK)$ to $O(N)$. We construct a large-scale dataset containing 196K samples with only 5% of the computational resources required by previous methods. Empirical results demonstrate that PRMs trained on our dataset consistently outperform existing automated annotation approaches on both Best-of-N strategy and ProcessBench.