Hybrid Retrieval-Augmented Generation for Robust Multilingual Document Question Answering

TL;DR

This paper presents a multilingual retrieval-augmented generation system tailored for question answering on noisy historical documents, effectively handling OCR errors, language variation, and temporal language drift.

Contribution

It introduces a modular, robust pipeline combining semantic query expansion, evidence-grounded generation prompts, and systematic component evaluation for improved historical document QA.

Findings

Enhanced retrieval robustness via Reciprocal Rank Fusion.

Faithful answers with explicit abstention for unanswerable questions.

Stable recall performance across query variations.

Abstract

Large-scale digitization initiatives have unlocked massive collections of historical newspapers, yet effective computational access remains hindered by OCR corruption, multilingual orthographic variation, and temporal language drift. We develop and evaluate a multilingual Retrieval-Augmented Generation pipeline specifically designed for question answering on noisy historical documents. Our approach integrates: (i) semantic query expansion and multi-query fusion using Reciprocal Rank Fusion to improve retrieval robustness against vocabulary mismatch; (ii) a carefully engineered generation prompt that enforces strict grounding in retrieved evidence and explicit abstention when evidence is insufficient; and (iii) a modular architecture enabling systematic component evaluation. We conduct comprehensive ablation studies on Named Entity Recognition and embedding model selection, demonstrating the importance of syntactic coherence in entity extraction and balanced performance-efficiency trade-offs in dense retrieval. Our end-to-end evaluation framework shows that the pipeline generates faithful answers for well-supported queries while correctly abstaining from unanswerable questions. The hybrid retrieval strategy improves recall stability, particularly benefiting from RRF's ability to smooth performance variance across query formulations. We release our code and configurations at https://anonymous.4open.science/r/RAGs-C5AE/, providing a reproducible foundation for robust historical document question answering.