When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning

TL;DR

This paper analyzes the internal mechanisms of large reasoning models trained with reinforcement learning, identifying three reasoning modes and their effects on efficiency and accuracy, to improve their reliability and performance.

Contribution

It introduces a mechanistic analysis of reasoning modes in LRMs, revealing how different thinking strategies impact efficiency and accuracy, and highlights the need for adaptive improvements.

Findings

NT reduces output length but lowers accuracy

ET and IT maintain accuracy with shorter responses

RL-optimized LRMs show fundamental inconsistencies

Abstract

Large reasoning models (LRMs) have significantly advanced performance on complex tasks, yet their tendency to overthink introduces inefficiencies. This study investigates the internal mechanisms of reinforcement learning (RL)-trained LRMs when prompted to save thinking, revealing three distinct thinking modes: no thinking (NT), explicit thinking (ET), and implicit thinking (IT). Through comprehensive analysis of confidence in thinking termination, attention from thinking to generation, and attentional focus on input sections, we uncover key factors influencing the reasoning behaviors. We further find that NT reduces output length at the cost of accuracy, while ET and IT maintain accuracy with reduced response length. Our findings expose fundamental inconsistencies in RL-optimized LRMs, necessitating adaptive improvements for reliable efficiency.