Confident Neural Network Regression with Bootstrapped Deep Ensembles

TL;DR

This paper introduces Bootstrapped Deep Ensembles, a computationally efficient extension of Deep Ensembles for regression, which explicitly accounts for classical data-induced uncertainty, leading to improved uncertainty estimation.

Contribution

It proposes a novel, cost-effective method that incorporates classical finite data uncertainty into neural network ensembles for better regression uncertainty estimation.

Findings

Significant improvement over standard Deep Ensembles in uncertainty estimation.

Method effectively captures classical data uncertainty in neural network regression.

Experimental results demonstrate enhanced predictive reliability.

Abstract

With the rise of the popularity and usage of neural networks, trustworthy uncertainty estimation is becoming increasingly essential. One of the most prominent uncertainty estimation methods is Deep Ensembles (Lakshminarayanan et al., 2017) . A classical parametric model has uncertainty in the parameters due to the fact that the data on which the model is build is a random sample. A modern neural network has an additional uncertainty component since the optimization of the network is random. Lakshminarayanan et al. (2017) noted that Deep Ensembles do not incorporate the classical uncertainty induced by the effect of finite data. In this paper, we present a computationally cheap extension of Deep Ensembles for the regression setting, called Bootstrapped Deep Ensembles, that explicitly takes this classical effect of finite data into account using a modified version of the parametric bootstrap. We demonstrate through an experimental study that our method significantly improves upon standard Deep Ensembles