Thinking with Nothinking Calibration: A New In-Context Learning Paradigm in Reasoning Large Language Models

TL;DR

This paper introduces JointThinking, a novel in-context learning paradigm for reasoning large language models that enhances reasoning accuracy and robustness by generating parallel answers and verifying consistency through a second reasoning step.

Contribution

The paper proposes a new ICL paradigm called JointThinking that prompts models to produce dual answers and verify consistency, improving reasoning performance especially on out-of-distribution tasks.

Findings

JointThinking outperforms chain-of-thought and voting methods on multiple benchmarks.

It achieves comparable in-distribution performance to training-based SOTA methods.

Performance improves with larger models and a second reasoning step.

Abstract

Reasoning large language models (RLLMs) have recently demonstrated remarkable capabilities through structured and multi-step reasoning. While prior research has primarily focused on improving their training and inference strategies, their potential for in-context learning (ICL) remains largely underexplored. To fill this gap, we propose Thinking with Nothinking Calibration (JointThinking), a new ICL paradigm that prompts the model to generate two answers in parallel: one in Thinking mode and the other in Nothinking mode. A second round of Thinking is triggered only when the two initial responses are inconsistent, using a single prompt with two different answers. Extensive experiments across multiple reasoning benchmarks demonstrate that JointThinking significantly outperforms few-shot chain-of-thought (CoT), thinking twice and majority voting. Moreover, it achieves comparable in-distribution performance to training-based SOTA reasoning method, while substantially outperforming on out-of-distribution tasks. We further conduct a systematic analysis of the calibration mechanism, showing the importance of structural thinking diversity and the benefits of consistency check. Additionally, we observe that the performance gap between actual and ideal reasoning narrows as model size increases in the second thinking, indicating the strong scalability of our approach. Finally, we discuss current limitations and outline promising directions for future ICL research in RLLMs.