MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answering

TL;DR

This paper introduces MRAG, a modular, trainless retrieval framework designed to improve time-sensitive question answering by effectively handling temporal reasoning, outperforming existing retrieval methods on a new benchmark with temporal perturbations.

Contribution

The paper proposes MRAG, a novel modular retrieval framework that enhances temporal reasoning in question answering without requiring training, and introduces TempRAGEval, a benchmark for evaluating temporal question answering.

Findings

MRAG significantly outperforms baseline retrievers on TempRAGEval.

All existing retrieval methods struggle with temporally complex questions.

MRAG improves final answer accuracy through better evidence retrieval and ranking.

Abstract

Understanding temporal relations and answering time-sensitive questions is crucial yet a challenging task for question-answering systems powered by large language models (LLMs). Existing approaches either update the parametric knowledge of LLMs with new facts, which is resource-intensive and often impractical, or integrate LLMs with external knowledge retrieval (i.e., retrieval-augmented generation). However, off-the-shelf retrievers often struggle to identify relevant documents that require intensive temporal reasoning. To systematically study time-sensitive question answering, we introduce the TempRAGEval benchmark, which repurposes existing datasets by incorporating temporal perturbations and gold evidence labels. As anticipated, all existing retrieval methods struggle with these temporal reasoning-intensive questions. We further propose Modular Retrieval (MRAG), a trainless framework that includes three modules: (1) Question Processing that decomposes question into a main content and a temporal constraint; (2) Retrieval and Summarization that retrieves evidence and uses LLMs to summarize according to the main content; (3) Semantic-Temporal Hybrid Ranking that scores each evidence summarization based on both semantic and temporal relevance. On TempRAGEval, MRAG significantly outperforms baseline retrievers in retrieval performance, leading to further improvements in final answer accuracy.