Calibrated and uncertain? Evaluating uncertainty estimates in binary classification models

TL;DR

This study evaluates six probabilistic machine learning algorithms for binary classification, focusing on their ability to produce well-calibrated and meaningful uncertainty estimates, especially for out-of-distribution data, using synthetic datasets.

Contribution

It provides a comparative analysis of various uncertainty estimation methods in deep learning, highlighting their strengths and limitations in calibration and out-of-distribution detection.

Findings

All algorithms showed good calibration on synthetic data.

Deep learning methods struggled with out-of-distribution uncertainty reflection.

None of the methods consistently detected lack of evidence for OOD data.

Abstract

Rigorous statistical methods, including parameter estimation with accompanying uncertainties, underpin the validity of scientific discovery, especially in the natural sciences. With increasingly complex data models such as deep learning techniques, uncertainty quantification has become exceedingly difficult and a plethora of techniques have been proposed. In this case study, we use the unifying framework of approximate Bayesian inference combined with empirical tests on carefully created synthetic classification datasets to investigate qualitative properties of six different probabilistic machine learning algorithms for class probability and uncertainty estimation: (i) a neural network ensemble, (ii) neural network ensemble with conflictual loss, (iii) evidential deep learning, (iv) a single neural network with Monte Carlo Dropout, (v) Gaussian process classification and (vi) a Dirichlet process mixture model. We check if the algorithms produce uncertainty estimates which reflect commonly desired properties, such as being well calibrated and exhibiting an increase in uncertainty for out-of-distribution data points. Our results indicate that all algorithms show reasonably good calibration performance on our synthetic test sets, but none of the deep learning based algorithms provide uncertainties that consistently reflect lack of experimental evidence for out-of-distribution data points. We hope our study may serve as a clarifying example for researchers that are using or developing methods of uncertainty estimation for scientific data-driven modeling and analysis.