Atom of Thoughts for Markov LLM Test-Time Scaling

TL;DR

This paper introduces Atom of Thoughts (), a Markovian reasoning framework that enhances test-time scaling efficiency in large language models by decomposing reasoning into atomic units, outperforming existing methods.

Contribution

The paper proposes a novel Markovian reasoning process that reduces redundant computation and enables scalable, high-performance inference in LLMs through atomic reasoning units.

Findings

 outperforms baselines with increased computational budgets

Seamless integration with various reasoning frameworks and LLMs

Uncovers emergent atomic reasoning structure in large models

Abstract

Large Language Models (LLMs) have achieved significant performance gains through test-time scaling methods. However, existing approaches often incur redundant computations due to the accumulation of historical dependency information during inference. To address this challenge, we leverage the memoryless property of Markov processes to minimize reliance on historical context and propose a Markovian reasoning process. This foundational Markov chain structure enables seamless integration with various test-time scaling methods, thereby improving their scaling efficiency. By further scaling up the Markovian reasoning chain through integration with techniques such as tree search and reflective refinement, we uncover an emergent atomic reasoning structure, where reasoning trajectories are decomposed into a series of self-contained, low-complexity atomic units. We name this design Atom of Thoughts (\our). Extensive experiments demonstrate that \our consistently outperforms existing baselines as computational budgets increase. Importantly, \our integrates seamlessly with existing reasoning frameworks and different LLMs (both reasoning and non-reasoning), facilitating scalable, high-performance inference.We submit our code alongside this paper and will make it publicly available to facilitate reproducibility and future research.