Accurate Uncertainty Estimation and Decomposition in Ensemble Learning

TL;DR

This paper presents a Bayesian nonparametric ensemble method that accurately estimates and decomposes model uncertainty, providing robust uncertainty quantification and insights into sources of noise and bias in ensemble predictions.

Contribution

It introduces a novel Bayesian nonparametric approach to enhance ensemble models with uncertainty estimation and decomposition capabilities, backed by theoretical guarantees.

Findings

Achieves accurate uncertainty estimates under complex noise conditions

Enables decomposition of predictive uncertainty into noise and error components

Demonstrates utility in air pollution exposure prediction

Abstract

Ensemble learning is a standard approach to building machine learning systems that capture complex phenomena in real-world data. An important aspect of these systems is the complete and valid quantification of model uncertainty. We introduce a Bayesian nonparametric ensemble (BNE) approach that augments an existing ensemble model to account for different sources of model uncertainty. BNE augments a model's prediction and distribution functions using Bayesian nonparametric machinery. It has a theoretical guarantee in that it robustly estimates the uncertainty patterns in the data distribution, and can decompose its overall predictive uncertainty into distinct components that are due to different sources of noise and error. We show that our method achieves accurate uncertainty estimates under complex observational noise, and illustrate its real-world utility in terms of uncertainty decomposition and model bias detection for an ensemble in predict air pollution exposures in Eastern Massachusetts, USA.