Step Back to Leap Forward: Self-Backtracking for Boosting Reasoning of Language Models

TL;DR

This paper introduces a self-backtracking mechanism for large language models, enabling autonomous search process internalization, which significantly improves reasoning performance and efficiency in LLMs, advancing toward more robust AGI reasoning.

Contribution

It proposes a novel self-backtracking method allowing LLMs to autonomously decide when to backtrack, improving reasoning and efficiency over existing methods.

Findings

Achieves over 40% performance improvement compared to supervised fine-tuning.

Enhances reasoning ability by enabling autonomous backtracking during inference.

Transforms slow-thinking into fast-thinking through self-improvement.

Abstract

The integration of slow-thinking mechanisms into large language models (LLMs) offers a promising way toward achieving Level 2 AGI Reasoners, as exemplified by systems like OpenAI's o1. However, several significant challenges remain, including inefficient overthinking and an overreliance on auxiliary reward models. We point out that these limitations stem from LLMs' inability to internalize the search process, a key component of effective reasoning. A critical step toward addressing this issue is enabling LLMs to autonomously determine when and where to backtrack, a fundamental operation in traditional search algorithms. To this end, we propose a self-backtracking mechanism that equips LLMs with the ability to backtrack during both training and inference. This mechanism not only enhances reasoning ability but also efficiency by transforming slow-thinking processes into fast-thinking through self-improvement. Empirical evaluations demonstrate that our proposal significantly enhances the reasoning capabilities of LLMs, achieving a performance gain of over 40 percent compared to the optimal-path supervised fine-tuning method. We believe this study introduces a novel and promising pathway for developing more advanced and robust Reasoners.