Think Straight, Stop Smart: Structured Reasoning for Efficient Multi-Hop RAG

TL;DR

TSSS introduces a structured multi-hop RAG framework that improves efficiency and stability by caching reasoning patterns and deterministically stopping when reasoning becomes repetitive, achieving state-of-the-art results.

Contribution

The paper presents TSSS, a novel structured reasoning and termination method that enhances efficiency and stability in multi-hop RAG tasks.

Findings

Achieves state-of-the-art accuracy on HotpotQA, 2WikiMultiHop, and MuSiQue.

Reduces token generation cost through template-based reasoning.

Enables faster inference with reliable stopping criteria.

Abstract

Multi-hop retrieval-augmented generation (RAG) is a promising strategy for complex reasoning, yet existing iterative prompting approaches remain inefficient. They often regenerate predictable token sequences at every step and rely on stochastic stopping, leading to excessive token usage and unstable termination. We propose TSSS (Think Straight, Stop Smart), a structured multi-hop RAG framework designed for efficiency. TSSS introduces (i) a template-based reasoning that caches recurring prefixes and anchors sub-queries to the main question, reducing token generation cost while promoting stable reasoning, and (ii) a retriever-based terminator, which deterministically halts reasoning once additional sub-queries collapse into repetition. This separation of structured reasoning and termination control enables both faster inference and more reliable answers. On HotpotQA, 2WikiMultiHop, and MuSiQue, TSSS achieves state-of-the-art accuracy and competitive efficiency among RAG-CoT approaches, highlighting its effectiveness in efficiency-constrained scenarios such as on-device inference.