Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning

TL;DR

This paper introduces a semantic-aware reinforcement learning approach to reduce internal and external redundancy in Chain-of-Thought reasoning traces, improving efficiency and interpretability without sacrificing accuracy.

Contribution

It proposes a dual-penalty framework targeting internal and external redundancy, with a sliding-window analysis and normalized metrics, enhancing reasoning trace conciseness and interpretability.

Findings

Significant compression of reasoning traces with minimal accuracy loss

External redundancy can be eliminated without performance impact

Internal redundancy removal requires careful calibration to preserve reasoning quality

Abstract

Large Reasoning Models (LRMs) often suffer from overthinking, generating verbose reasoning traces that compromise both computational efficiency and interpretability. Unlike prior efforts that rely on global length-based rewards, we propose a semantic-aware decomposition of redundancy into two distinct forms: internal redundancy (informational stagnation within the reasoning process) and external redundancy (superfluous continuation after the final answer). We introduce a dual-penalty reinforcement learning framework that surgically targets these inefficiencies: a sliding-window semantic analysis is employed to penalize low-gain steps within the reasoning trajectory, while a normalized metric suppresses the post-answer tail. Extensive experiments demonstrate that our method significantly compresses Chain-of-Thought traces with minimal accuracy degradation, while maintaining strong generalization to out-of-domain tasks. Crucially, we reveal an asymmetry in redundancy: external redundancy can be safely eliminated without performance loss, whereas internal redundancy removal requires a calibrated trade-off to maintain reasoning fidelity. Our framework enables fine-grained, implicit control over reasoning length, paving the way for more concise and interpretable LRMs.