Quantifying Uncertainty in Deep Spatiotemporal Forecasting

TL;DR

This paper systematically evaluates uncertainty quantification methods for deep spatiotemporal forecasting, highlighting their statistical and computational trade-offs across real-world applications.

Contribution

It provides a unified framework for analyzing Bayesian and frequentist UQ methods in spatiotemporal forecasting, with extensive empirical comparisons.

Findings

Bayesian methods are more robust in mean prediction.

Frequentist confidence levels offer better coverage.

Quantile regression is computationally efficient for single intervals.

Abstract

Deep learning is gaining increasing popularity for spatiotemporal forecasting. However, prior works have mostly focused on point estimates without quantifying the uncertainty of the predictions. In high stakes domains, being able to generate probabilistic forecasts with confidence intervals is critical to risk assessment and decision making. Hence, a systematic study of uncertainty quantification (UQ) methods for spatiotemporal forecasting is missing in the community. In this paper, we describe two types of spatiotemporal forecasting problems: regular grid-based and graph-based. Then we analyze UQ methods from both the Bayesian and the frequentist point of view, casting in a unified framework via statistical decision theory. Through extensive experiments on real-world road network traffic, epidemics, and air quality forecasting tasks, we reveal the statistical and computational trade-offs for different UQ methods: Bayesian methods are typically more robust in mean prediction, while confidence levels obtained from frequentist methods provide more extensive coverage over data variations. Computationally, quantile regression type methods are cheaper for a single confidence interval but require re-training for different intervals. Sampling based methods generate samples that can form multiple confidence intervals, albeit at a higher computational cost.