Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoring

TL;DR

This paper introduces a novel early-exit method for large reasoning language models that monitors reasoning path deviations via high-entropy tokens to effectively mitigate overthinking, improving performance and efficiency.

Contribution

The proposed method couples early-exit with native reasoning by using a path deviation index based on high-entropy tokens, reducing overthinking without extra training overhead.

Findings

Significant performance gains over vanilla Chain-of-Thought methods.

Effective detection and termination of overthinking trajectories.

Improved efficiency in reasoning tasks across multiple benchmarks.

Abstract

Large Reasoning Language Models (LRLMs) demonstrate impressive capabilities on complex tasks by utilizing long Chain-of-Thought reasoning. However, they are prone to overthinking, which generates redundant reasoning steps that degrade both performance and efficiency. Recently, early-exit strategies are proposed to mitigate overthinking by dynamically and adaptively terminating redundant reasoning. However, current early-exit methods either introduce extra training overhead by relying on proxy models or limit inference throughput due to the frequent content switching between reasoning and generating probing answers. Moreover, most early-exit methods harm LRLMs performance due to over-truncation. Our insight stems from an observation: overthinking often causes LRLMs to deviate from the correct reasoning path, which is frequently accompanied by high-entropy transition tokens. Given this, we propose an early-exit method deeply coupled with the native reasoning process, which leverages the path deviation index as a dedicated monitoring metric for the frequent occurrence of high-entropy transition tokens to dynamically detect and terminate overthinking trajectories. We conduct experiments across multiple benchmarks using LRLMs of different types and scales, and the results indicate that our method delivers the largest performance improvement over vanilla CoT compared to existing early-exit methods.