Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Ones

TL;DR

This paper investigates the trade-offs between sequential and parallel inference strategies in large language models, demonstrating that in certain graph reasoning tasks, longer chains of thought can exponentially outperform multiple short chains.

Contribution

It reveals conditions where sequential reasoning with longer chains offers exponential benefits over parallel approaches, supported by theoretical analysis and extensive experiments.

Findings

Sequential scaling can exponentially outperform parallel scaling in specific graph reasoning tasks.

Longer chain-of-thought strategies improve reasoning accuracy in challenging graph connectivity problems.

Experimental results validate the theoretical advantage of sequential reasoning across various models.

Abstract

Inference-time computation has emerged as a promising scaling axis for improving large language model reasoning. However, despite yielding impressive performance, the optimal allocation of inference-time computation remains poorly understood. A central question is whether to prioritize sequential scaling (e.g., longer chains of thought) or parallel scaling (e.g., majority voting across multiple short chains of thought). In this work, we seek to illuminate the landscape of test-time scaling by demonstrating the existence of reasoning settings where sequential scaling offers an exponential advantage over parallel scaling. These settings are based on graph connectivity problems in challenging distributions of graphs. We validate our theoretical findings with comprehensive experiments across a range of language models, including models trained from scratch for graph connectivity with different chain of thought strategies as well as large reasoning models.