SpatialEpiBench: Benchmarking Spatial Information and Epidemic Priors in Forecasting

TL;DR

SpatialEpiBench provides a standardized, realistic benchmark for evaluating spatiotemporal epidemic forecasting models, revealing current methods often underperform simple baselines and highlighting key challenges.

Contribution

It introduces a comprehensive benchmark with datasets, evaluation protocols, and insights into the limitations of existing spatial epidemic forecasting models.

Findings

Most models underperform a simple last-value baseline.

Major failure modes include poor outbreak anticipation and handling noise.

Limited utility of geographic adjacency in epidemiological predictions.

Abstract

Accurate epidemic forecasting is crucial for public health response, resource allocation, and outbreak intervention, but remains difficult with sparse, noisy, and highly non-stationary data. Because epidemics unfold across interacting regions, spatiotemporal methods are natural candidates for improving forecasts. Despite growing interest in spatial information, no standardized benchmark exists, and current evaluations often use simple chronological train-test splits that do not reflect real-time forecasting practice. We address this gap with SpatialEpiBench, a challenging benchmark for spatiotemporal epidemic forecasting in realistic public-health settings. SpatialEpiBench includes 11 epidemic datasets with standardized rolling evaluations and outbreak-specific metrics. We evaluate adjacency-informed forecasting models with widely used epidemic priors that adapt general models to epidemiology, but find that most methods underperform a simple last-value baseline from 1 day to 1 month ahead, even during outbreaks and with these priors. We identify three major failure modes: (1) poor outbreak anticipation, (2) difficulty handling sparsity and noise, and (3) limited utility of common geographic adjacency for epidemiological spatial information. We release benchmark data, code, and instructions at https://github.com/Rachel-Lyu/SpatialEpiBench to support development of operationally useful epidemic forecasting models.