The Art of Efficient Reasoning: Data, Reward, and Optimization

TL;DR

This paper systematically investigates efficient reasoning in large language models, emphasizing reward shaping, length adaptation, and training strategies to improve reasoning accuracy while reducing computational costs.

Contribution

It provides a comprehensive analysis of the mechanics of efficient reasoning, including a two-stage training paradigm and practical guidelines validated across multiple models.

Findings

Maintaining positive reward density prevents the short-is-correct trap.

Learned length bias generalizes across domains and difficulty levels.

Extensive experiments validate the robustness of proposed strategies.

Abstract

Large Language Models (LLMs) consistently benefit from scaled Chain-of-Thought (CoT) reasoning, but also suffer from heavy computational overhead. To address this issue, efficient reasoning aims to incentivize short yet accurate thinking trajectories, typically through reward shaping with Reinforcement Learning (RL). In this paper, we systematically investigate the mechanics of efficient reasoning for LLMs. For comprehensive evaluation, we advocate for more fine-grained metrics, including length distribution conditioned on correctness and performance across a wide spectrum of token budgets ranging from 2k to 32k. First, we reveal that the training process follows a two-stage paradigm: length adaptation and reasoning refinement. Through extensive experiments (about 0.2 million GPU hours) in a unified protocol, we deconstruct training prompts and rollouts, reward shaping, and optimization strategies. A central finding is to maintain a sufficient density of positive reward signals and avoid the short-is-correct trap. Moreover, the learned length bias generalizes across domains and difficulty levels. We distill these findings into valuable insights and practical guidelines, and validate them across the Qwen3 models ranging from 0.6B to 30B, demonstrating the robustness and generalization. Weights are available at https://wutaiqiang.github.io/project/Art