CheXTemporal: A Dataset for Temporally-Grounded Reasoning in Chest Radiography

TL;DR

CheXTemporal introduces a new dataset with detailed temporal and spatial annotations for chest X-ray analysis, enabling improved reasoning about disease progression over time.

Contribution

The paper presents CheXTemporal, a comprehensive dataset with annotations for temporal and spatial reasoning in chest radiographs, and evaluates current models' limitations in this context.

Findings

Current models struggle with spatial grounding and fine-grained temporal reasoning.

Models perform better on prominent progression categories like 'worse' than on subtle states like 'stable'.

Models show limited robustness under distribution shifts.

Abstract

Chest radiograph interpretation requires temporal reasoning over prior and current studies, yet most vision-language models are trained on static image-report pairs and lack explicit supervision for modeling longitudinal change. We introduce CheXTemporal, a dataset for temporally grounded reasoning in chest radiography consisting of paired prior-current chest X-rays (CXR) with finding-level temporal and spatial annotations. The dataset includes a five-class progression taxonomy (new, worse, stable, improved, resolved), localized spatial supervision of pathology, explicit spatial-temporal alignment across paired studies, and multi-source coverage for cross-domain evaluation. We additionally construct a 280K-pair silver dataset with automatically derived temporal and anatomical supervision for large-scale evaluation under weaker supervision. Using these resources, we evaluate multiple state-of-the-art vision-language CXR models on grounding and progression-classification tasks in a zero-shot setting. Across both gold and silver evaluations, current models exhibit consistent limitations in spatial grounding, fine-grained temporal reasoning, and robustness under distribution shift. In particular, models perform substantially better on salient progression categories such as worse than on temporally subtle states such as stable and resolved, suggesting limited modeling of longitudinal disease evolution in chest radiography.