RASTeR: Robust, Agentic, and Structured Temporal Reasoning

TL;DR

RASTeR is a prompting framework that enhances large language models' ability to perform reliable temporal question answering by evaluating context relevance, constructing temporal knowledge graphs, and correcting inconsistencies, especially under noisy or outdated information.

Contribution

The paper introduces RASTeR, a novel prompting framework that separates context evaluation from answer generation, improving robustness in temporal reasoning tasks for LLMs.

Findings

RASTeR improves accuracy by over 12% compared to baselines.

It maintains 75% accuracy with 40 distractors in noisy contexts.

RASTeR outperforms existing methods across multiple datasets and models.

Abstract

Temporal question answering (TQA) remains a challenge for large language models (LLMs), particularly when retrieved content may be irrelevant, outdated, or temporally inconsistent. This is especially critical in applications like clinical event ordering, and policy tracking, which require reliable temporal reasoning even under noisy or outdated information. To address this challenge, we introduce RASTeR: \textbf{R}obust, \textbf{A}gentic, and \textbf{S}tructured, \textbf{Te}mporal \textbf{R}easoning, a prompting framework that separates context evaluation from answer generation. RASTeR first assesses the relevance and temporal coherence of the retrieved context, then constructs a temporal knolwedge graph (TKG) to better facilitate reasoning. When inconsistencies are detected, RASTeR selectively corrects or discards context before generating an answer. Across multiple datasets and LLMs, RASTeR consistently improves robustness\footnote{\ Some TQA work defines robustness as handling diverse temporal phenomena. Here, we define it as the ability to answer correctly despite suboptimal context}. We further validate our approach through a ``needle-in-the-haystack'' study, in which relevant context is buried among distractors. With forty distractors, RASTeR achieves 75\% accuracy, over 12\% ahead of the runner up