Beyond Model Scaling: Test-Time Intervention for Efficient Deep Reasoning

TL;DR

This paper introduces Think-with-Me, a test-time interactive reasoning paradigm that incorporates external feedback to improve the efficiency and accuracy of large reasoning models during multi-step reasoning tasks.

Contribution

It proposes a novel test-time intervention method using external feedback at transitional points to enhance reasoning efficiency and accuracy in large models.

Findings

Outperforms existing methods on AIME24 with 7.19% higher accuracy.

Reduces reasoning length by 81% under limited context windows.

Effective in security and creative reasoning tasks.

Abstract

Large Reasoning Models (LRMs) excel at multi-step reasoning but often suffer from inefficient reasoning processes like overthinking and overshoot, where excessive or misdirected reasoning increases computational cost and degrades performance. Existing efficient reasoning methods operate in a closed-loop manner, lacking mechanisms for external intervention to guide the reasoning process. To address this, we propose Think-with-Me, a novel test-time interactive reasoning paradigm that introduces external feedback intervention into the reasoning process. Our key insights are that transitional conjunctions serve as natural points for intervention, signaling phases of self-validation or exploration and using transitional words appropriately to prolong the reasoning enhances performance, while excessive use affects performance. Building on these insights, Think-with-Me pauses reasoning at these points for external feedback, adaptively extending or terminating reasoning to reduce redundancy while preserving accuracy. The feedback is generated via a multi-criteria evaluation (rationality and completeness) and comes from either human or LLM proxies. We train the target model using Group Relative Policy Optimization (GRPO) to adapt to this interactive mode. Experiments show that Think-with-Me achieves a superior balance between accuracy and reasoning length under limited context windows. On AIME24, Think-with-Me outperforms QwQ-32B by 7.19% in accuracy while reducing average reasoning length by 81% under an 8K window. The paradigm also benefits security and creative tasks.