When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling

TL;DR

This paper investigates the diminishing returns of extended reasoning in large language models, revealing overthinking issues and proposing cost-effective stopping strategies based on problem difficulty.

Contribution

It systematically analyzes the utility of additional reasoning tokens, highlighting overthinking phenomena and advocating for adaptive compute allocation strategies.

Findings

Marginal utility of reasoning diminishes at higher compute budgets.

Overthinking can lead to abandoning correct answers.

Moderate reasoning budgets can achieve similar accuracy with less computation.

Abstract

Scaling test-time compute through extended chains of thought has become a dominant paradigm for improving large language model reasoning. However, existing research implicitly assumes that longer thinking always yields better results. This assumption remains largely unexamined. We systematically investigate how the marginal utility of additional reasoning tokens changes as compute budgets increase. We find that marginal returns diminish substantially at higher budgets and that models exhibit ``overthinking'', where extended reasoning is associated with abandoning previously correct answers. Furthermore, we show that optimal thinking length varies across problem difficulty, suggesting that uniform compute allocation is suboptimal. Our cost-aware evaluation framework reveals that stopping at moderate budgets can reduce computation significantly while maintaining comparable accuracy.