TEMPO: Transformers for Temporal Disease Progression from Cross-Sectional Data

TL;DR

Tempo is a Transformer-based model that learns both ordinal and continuous disease progression sequences from cross-sectional data, outperforming existing methods in synthetic benchmarks and revealing plausible Alzheimer's progression patterns.

Contribution

Introduces Tempo, a novel Transformer architecture that infers disease progression from cross-sectional data without custom inference algorithms, improving accuracy and interpretability.

Findings

Tempo reduces Kendall's Tau distance by 52.89% on synthetic benchmarks.

Tempo decreases staging MAE by 25.33% compared to state-of-the-art.

Applied to ADNI, Tempo recovers plausible Alzheimer's disease progression stages.

Abstract

Event-Based Models (EBMs) infer biomarker progression from cross-sectional data but typically only as ordinal sequences and rely on rigid model assumptions. We propose \textsc{Tempo}, a Transformer architecture that learns both ordinal and continuous event sequences through simulation-based supervised learning. \textsc{Tempo} uses two Transformer modules: one treats biomarkers as tokens to infer event sequencing; the other treats patients as tokens, representing each by their per-biomarker abnormality profile, to infer patients' disease stages. On synthetic benchmarks, \textsc{Tempo} reduces normalized Kendall's Tau distance by 52.89\% and staging MAE by 25.33\% compared to state-of-the-art SA-EBM, with larger reductions in high-dimensional settings (58.88\% and 61.10\%). Applied to ADNI, \textsc{Tempo} recovers a biologically plausible Alzheimer's progression: early medial temporal atrophy, followed by amyloid accumulation and cognitive decline, and late-stage tau pathology with terminal acceleration of global neurodegeneration -- broadly consistent with established disease models. \textsc{Tempo} also eliminates the need to derive custom inference algorithms and enables rapid empirical comparison of generative hypotheses.