SmartSwitch: Advancing LLM Reasoning by Overcoming Underthinking via Promoting Deeper Thought Exploration

TL;DR

SmartSwitch is a reasoning framework that improves large language models' complex reasoning by detecting underthinking, encouraging deeper exploration, and significantly boosting performance on mathematical reasoning benchmarks.

Contribution

It introduces a plug-and-play inference strategy that monitors and guides LLM reasoning to overcome shallow thinking and enhance problem-solving capabilities.

Findings

Significant performance improvements on mathematical reasoning benchmarks.

Effective detection and correction of underthinking during inference.

Applicable to various large language model sizes.

Abstract

The long chain-of-thought (LongCoT) capability is central to the recent breakthroughs achieved by large language models in complex reasoning tasks. However, the accompanying issue of ''underthinking'', where models exhibit shallow reasoning by frequently switching thoughts without sufficient exploration, limits both performance and token efficiency. To address this problem, we propose a simple yet effective reasoning strategy: the SmartSwitch inference framework. This framework can be easily integrated into any large language model as a plug-and-play solution, continuously monitoring the model's reasoning process to detect underthinking and guide it toward deeper exploration of promising but overlooked thoughts. Specifically, the perception module identifies points where thoughts switch and evaluates the potential of the preceding thought using an off-the-shelf process reward model (PRM). If a high-potential thought is found to be prematurely abandoned, the intervention module interrupts the ongoing inference, backtracks to the point before the switch, and inserts a "deepening prompt" to encourage further exploration along that promising path. Extensive experiments on challenging mathematical reasoning benchmarks demonstrate that our method significantly enhances the performance of various large language models of different sizes.