QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation

TL;DR

QuestA introduces question augmentation during RL training to enhance reasoning in large language models, leading to state-of-the-art results on math benchmarks with 1.5B models.

Contribution

The paper proposes a simple question augmentation strategy during RL training that improves reasoning performance of LLMs on math tasks.

Findings

Achieved new state-of-the-art results on math benchmarks.

Improved pass@1 and pass@k metrics on challenging problems.

Enhanced reasoning capabilities of open-source models.

Abstract

Reinforcement learning (RL) has emerged as a central paradigm for training large language models (LLMs) in reasoning tasks. Yet recent studies question RL's ability to incentivize reasoning capacity beyond the base model. This raises a key challenge: how can RL be adapted to solve harder reasoning problems more effectively? To address this challenge, we propose a simple yet effective strategy via Question Augmentation: introduce partial solutions during training to reduce problem difficulty and provide more informative learning signals. Our method, QuestA, when applied during RL training on math reasoning tasks, not only improves pass@1 but also pass@k-particularly on problems where standard RL struggles to make progress. This enables continual improvement over strong open-source models such as DeepScaleR and OpenMath Nemotron, further enhancing their reasoning capabilities. We achieve new state-of-the-art results on math benchmarks using 1.5B-parameter models: 72.50% (+10.73%) on AIME24, 62.29% (+12.79%) on AIME25, and 41.67% (+10.11%) on HMMT25. Code, data and model are available at https://github.com/foreverlasting1202/QuestA.