Lost at the Beginning of Reasoning

TL;DR

This paper demonstrates that the initial reasoning step in large language models heavily influences overall accuracy, and proposes a cost-effective sampling method to improve reasoning quality by focusing on the first step.

Contribution

It reveals the critical impact of the first reasoning step on model predictions and introduces an efficient sampling strategy to enhance reasoning quality and reduce inference costs.

Findings

Errors at the first reasoning step significantly degrade final predictions

The proposed sampling strategy reduces inference cost by up to 70%

High-quality initial reasoning steps lead to better overall reasoning outcomes

Abstract

Recent advancements in large language models (LLMs) have significantly advanced complex reasoning capabilities, particularly through extended chain-of-thought (CoT) reasoning that incorporates mechanisms such as backtracking, self-reflection, and self-correction. Despite these developments, the self-correction abilities of LLMs during long CoT reasoning remain underexplored. And recent findings on overthinking suggest that such models often engage in unnecessarily redundant reasoning. In this work, we empirically show that the first reasoning step exerts a disproportionately large influence on the final prediction. I.e., errors introduced at this stage can substantially degrade subsequent reasoning quality. This phenomenon is consistently observed across various state-of-the-art open- and closed-source reasoning models. Leveraging this insight, we propose an efficient sampling strategy that leverages a reward model to identify and retain high-quality first reasoning steps while discarding suboptimal ones, achieving up to a 70% reduction in inference cost without sacrificing any accuracy. Our work highlights the central role of the first reasoning step in generating a high-quality reasoning trajectory, and thus enabling significantly efficient sampling.