The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis

TL;DR

This paper investigates overthinking in large language models during reasoning, analyzing dynamics to identify optimal stopping points, and proposes an early-exit method that reduces token usage without sacrificing accuracy.

Contribution

It introduces the concept of Reasoning Completion Point (RCP) and develops RCPD, an inference-time early-exit method based on reasoning dynamics to improve efficiency.

Findings

RCPD reduces token usage by up to 44%.

Reasoning dynamics reveal a trade-off leading to redundancy.

The method maintains accuracy while improving efficiency.

Abstract

Test-time scaling via explicit reasoning trajectories significantly boosts large language model (LLM) performance but often triggers overthinking. To explore this, we analyze reasoning through two lenses: Reasoning Length Dynamics, which reveals a compensatory trade-off between thinking and answer content length that eventually leads to thinking redundancy, and Reasoning Semantic Dynamics, which identifies semantic convergence and repetitive oscillations. These dynamics uncover an instance-specific Reasoning Completion Point (RCP), beyond which computation continues without further performance gain. Since the RCP varies across instances, we propose a Reasoning Completion Point Detector (RCPD), an inference-time early-exit method that identifies the RCP by monitoring the rank dynamics of termination tokens (e.g., </think>). Across AIME and GPQA benchmarks using Qwen3 and DeepSeek-R1, RCPD reduces token usage by up to 44% while preserving accuracy, offering a principled approach to efficient test-time scaling.