FB-RAG: Improving RAG with Forward and Backward Lookup
Kushal Chawla, Alfy Samuel, Anoop Kumar, Daben Liu

TL;DR
FB-RAG introduces a forward-backward lookup strategy that enhances retrieval-augmented generation by leveraging future generation insights, leading to improved accuracy and reduced latency without complex fine-tuning.
Contribution
The paper presents a training-free, forward-looking framework for RAG that improves relevance and efficiency by using evidence from multiple outputs to guide context selection.
Findings
Consistently outperforms baselines across 9 datasets.
Achieves over 48% latency reduction on EN.QA.
Guides final model effectively even when forward-looking LLM fails.
Abstract
Traditional Retrieval-Augmented Generation (RAG) struggles with complex queries that lack strong signals to retrieve the most relevant context, forcing a trade-off between choosing a small context that misses key information and a large context that confuses the LLM. To address this, we propose Forward-Backward RAG (FB-RAG), a new training-free framework based on a simple yet powerful forward-looking strategy. FB-RAG employs a light-weight LLM to peek into potential future generations, using evidence from multiple sampled outputs to precisely identify the most relevant context for a final, more powerful generator. This improves performance without complex finetuning or Reinforcement Learning common in prior work. Across datasets from LongBench and Bench, FB-RAG consistently delivers strong results. Further, the performance gains can be achieved with reduced latency due to a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
