Rethinking External Slow-Thinking: From Snowball Errors to Probability of Correct Reasoning

TL;DR

This paper investigates the theoretical foundations of external slow-thinking in large language models, linking snowball errors to reasoning correctness and comparing various approaches to improve multi-step reasoning.

Contribution

It provides a theoretical analysis of slow-thinking mechanisms, connecting snowball errors to reasoning probability, and offers a comparative analysis of different external slow-thinking methods.

Findings

External slow-thinking mitigates error probability in reasoning.

Expanding search scope or internal reasoning capacity can improve performance.

The effectiveness of slow-thinking methods is not solely dependent on their framework.

Abstract

Test-time scaling, which is also often referred to as slow-thinking, has been demonstrated to enhance multi-step reasoning in large language models (LLMs). However, despite its widespread utilization, the mechanisms underlying slow-thinking methods remain poorly understood. This paper explores the mechanisms of external slow-thinking from a theoretical standpoint. We begin by examining the snowball error effect within the LLM reasoning process and connect it to the likelihood of correct reasoning using information theory. Building on this, we show that external slow-thinking methods can be interpreted as strategies to mitigate the error probability. We further provide a comparative analysis of popular external slow-thinking approaches, ranging from simple to complex, highlighting their differences and interrelationships. Our findings suggest that the efficacy of these methods is not primarily determined by the specific framework employed, and that expanding the search scope or the model's internal reasoning capacity may yield more sustained improvements in the long term. We open-source our code at https://github.com/ZyGan1999/Snowball-Errors-and-Probability.