Adaptive Ensemble Learning of Spatiotemporal Processes with Calibrated Predictive Uncertainty: A Bayesian Nonparametric Approach

TL;DR

This paper introduces a Bayesian nonparametric ensemble learning method that adaptively combines models based on their accuracy, providing calibrated uncertainty estimates for predictions in spatiotemporal data.

Contribution

It develops a probabilistic ensemble framework using Gaussian processes to adaptively weight models and accurately quantify predictive uncertainty, improving over traditional fixed-weight ensembles.

Findings

Method accurately captures model uncertainty in simulations.

Applied to spatial data for fine particle levels, demonstrating practical utility.

Provides interpretable uncertainty estimates for ensemble predictions.

Abstract

Ensemble learning is a mainstay in modern data science practice. Conventional ensemble algorithms assign to base models a set of deterministic, constant model weights that (1) do not fully account for individual models' varying accuracy across data subgroups, nor (2) provide uncertainty estimates for the ensemble prediction. These shortcomings can yield predictions that are precise but biased, which can negatively impact the performance of the algorithm in real-word applications. In this work, we present an adaptive, probabilistic approach to ensemble learning using a transformed Gaussian process as a prior for the ensemble weights. Given input features, our method optimally combines base models based on their predictive accuracy in the feature space, and provides interpretable estimates of the uncertainty associated with both model selection, as reflected by the ensemble weights, and the overall ensemble predictions. Furthermore, to ensure that this quantification of the model uncertainty is accurate, we propose additional machinery to non-parametrically model the ensemble's predictive cumulative density function (CDF) so that it is consistent with the empirical distribution of the data. We apply the proposed method to data simulated from a nonlinear regression model, and to generate a spatial prediction model and associated prediction uncertainties for fine particle levels in eastern Massachusetts, USA.