HiFi-RAG: Hierarchical Content Filtering and Two-Pass Generation for Open-Domain RAG

TL;DR

HiFi-RAG is a hierarchical retrieval and generation system that improves open-domain question answering by multi-stage filtering and two-pass answer generation, outperforming baselines on multiple datasets and metrics.

Contribution

The paper introduces HiFi-RAG, a novel multi-stage pipeline combining hierarchical content filtering and two-pass generation, leveraging different models for efficiency and reasoning.

Findings

Outperforms baseline in ROUGE-L and DeBERTaScore on validation set.

Achieves significant improvements on post-cutoff knowledge questions.

Utilizes cost-effective Gemini 2.5 models for different pipeline stages.

Abstract

Retrieval-Augmented Generation (RAG) in open-domain settings faces significant challenges regarding irrelevant information in retrieved documents and the alignment of generated answers with user intent. We present HiFi-RAG (Hierarchical Filtering RAG), the winning closed-source system in the Text-to-Text static evaluation of the MMU-RAGent NeurIPS 2025 Competition. Our approach moves beyond standard embedding-based retrieval via a multi-stage pipeline. We leverage the speed and cost-efficiency of Gemini 2.5 Flash (4-6x cheaper than Pro) for query formulation, hierarchical content filtering, and citation attribution, while reserving the reasoning capabilities of Gemini 2.5 Pro for final answer generation. On the MMU-RAGent validation set, our system outperformed the baseline, improving ROUGE-L to 0.274 (+19.6%) and DeBERTaScore to 0.677 (+6.2%). On Test2025, our custom dataset evaluating questions that require post-cutoff knowledge (post January 2025), HiFi-RAG outperforms the parametric baseline by 57.4% in ROUGE-L and 14.9% in DeBERTaScore.