Guideline Forest: Retrieval-Augmented Reasoning with Branching Experience-Induced Guidelines

TL;DR

Guideline Forest introduces a retrieval-augmented reasoning framework that stores and reuses high-quality reasoning traces to improve multi-step reasoning in large language models across various benchmarks.

Contribution

It presents a novel framework that explicitly leverages experience to guide reasoning, enabling controlled branching, aggregation, and improved performance over existing methods.

Findings

Consistent improvements on mathematical and programming benchmarks.

Experience retrieval and guideline-induced diversity are crucial for effectiveness.

Framework generalizes to enhance diverse reasoning paradigms and multi-model collaboration.

Abstract

Retrieval-augmented generation (RAG) has been widely adopted to ground large language models (LLMs) in external knowledge, yet it remains largely underexplored for improving reasoning. Existing methods either rely on online exploration during inference or heuristic supervision over reasoning trajectories, but they fail to effectively accumulate and reuse past reasoning experience. We propose Guideline Forest, a retrieval-augmented reasoning framework that explicitly leverages experience to guide multi-step reasoning. The framework stores high-quality, label-consistent reasoning traces as reusable memory, retrieves relevant experiences for new problems, and induces them into structured guidelines that steer reasoning and enable controlled branching and aggregation. Experiments on mathematical (GSM8K, MATH-500) and programming (MBPP, HumanEval) benchmarks demonstrate consistent improvements over strong reasoning baselines, including CoT, ReAct, ToT, FoT, and AFlow. Further analyses show that experience retrieval, guideline-induced diversity, and stepwise aggregation are key to the framework's effectiveness. Beyond single-model reasoning, Guideline Forest generalizes to enhance diverse reasoning paradigms and supports multi-model collaboration, highlighting its flexibility and scalability.