e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMs

TL;DR

This paper introduces e3, a training method that enables large language models to better explore and extrapolate reasoning capabilities at test time by chaining operations, leveraging gradients, and curriculum design.

Contribution

The paper proposes a novel training recipe e3 that enhances in-context exploration and extrapolation in LLMs, achieving state-of-the-art results on reasoning benchmarks.

Findings

e3-1.7B model achieves top scores on AIME'25 and HMMT'25.

e3 enables models to extrapolate to twice the training token budget.

Improved pass@k performance over base models.

Abstract

Test-time scaling offers a promising path to improve LLM reasoning by utilizing more compute at inference time; however, the true promise of this paradigm lies in extrapolation (i.e., improvement in performance on hard problems as LLMs keep "thinking" for longer, beyond the maximum token budget they were trained on). Surprisingly, we find that most existing reasoning models do not extrapolate well. We show that one way to enable extrapolation is by training the LLM to perform in-context exploration: training the LLM to effectively spend its test time budget by chaining operations (such as generation, verification, refinement, etc.), or testing multiple hypotheses before it commits to an answer. To enable in-context exploration, we identify three key ingredients as part of our recipe e3: (1) chaining skills that the base LLM has asymmetric competence in, e.g., chaining verification (easy) with generation (hard), as a way to implement in-context search; (2) leveraging "negative" gradients from incorrect traces to amplify exploration during RL, resulting in longer search traces that chains additional asymmetries; and (3) coupling task difficulty with training token budget during training via a specifically-designed curriculum to structure in-context exploration. Our recipe e3 produces the best known 1.7B model according to AIME'25 and HMMT'25 scores, and extrapolates to 2x the training token budget. Our e3-1.7B model not only attains high pass@1 scores, but also improves pass@k over the base model.