SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoning

TL;DR

SelfBudgeter is a self-adaptive reasoning strategy for large language models that dynamically allocates token budgets, reducing resource waste and latency while maintaining accuracy.

Contribution

It introduces a novel self-estimation and budget-guided reinforcement learning approach for efficient, controllable reasoning in large models.

Findings

Achieves 61% reduction in output length on math reasoning tasks.

Maintains accuracy while reducing token usage.

Enables user control over reasoning length and response time.

Abstract

Recently, large reasoning models demonstrate exceptional performance on various tasks. However, reasoning models always consume excessive tokens even for simple queries, leading to resource waste and prolonged user latency. To address this challenge, we propose SelfBudgeter - a self-adaptive reasoning strategy for efficient and controllable reasoning. Specifically, we first train the model to self-estimate the required reasoning budget based on the query. We then introduce budget-guided GRPO for reinforcement learning, which effectively maintains accuracy while reducing output length. Experimental results demonstrate that SelfBudgeter dynamically allocates budgets according to problem complexity, achieving an average response length compression of 61% on math reasoning tasks while maintaining accuracy. Furthermore, SelfBudgeter allows users to see how long generation will take and decide whether to continue or stop. Additionally, users can directly control the reasoning length by setting token budgets upfront.