Reading Between the Timelines: RAG for Answering Diachronic Questions

TL;DR

This paper introduces a temporal-aware RAG framework that improves longitudinal question answering by incorporating temporal logic and relevance, validated on a new challenging benchmark with significant accuracy gains.

Contribution

It proposes a novel RAG pipeline redesign with temporal logic integration and introduces the ADQAB benchmark for evaluating diachronic question answering.

Findings

Achieves 13-27% improvement over standard RAG.

Introduces the ADQAB benchmark for diachronic QA.

Demonstrates effective temporal relevance calibration.

Abstract

While Retrieval-Augmented Generation (RAG) excels at injecting static, factual knowledge into Large Language Models (LLMs), it exhibits a critical deficit in handling longitudinal queries that require tracking entities and phenomena across time. This blind spot arises because conventional, semantically-driven retrieval methods are not equipped to gather evidence that is both topically relevant and temporally coherent for a specified duration. We address this challenge by proposing a new framework that fundamentally redesigns the RAG pipeline to infuse temporal logic. Our methodology begins by disentangling a user's query into its core subject and its temporal window. It then employs a specialized retriever that calibrates semantic matching against temporal relevance, ensuring the collection of a contiguous evidence set that spans the entire queried period. To enable rigorous evaluation of this capability, we also introduce the Analytical Diachronic Question Answering Benchmark (ADQAB), a challenging evaluation suite grounded in a hybrid corpus of real and synthetic financial news. Empirical results on ADQAB show that our approach yields substantial gains in answer accuracy, surpassing standard RAG implementations by 13% to 27%. This work provides a validated pathway toward RAG systems capable of performing the nuanced, evolutionary analysis required for complex, real-world questions. The dataset and code for this study are publicly available at https://github.com/kwunhang/TA-RAG.