A Comparison of the Delta Method and the Bootstrap in Deep Learning Classification

TL;DR

This paper compares the Delta method and Bootstrap for quantifying predictive uncertainty in deep learning classifiers, showing a strong correlation and significant computational efficiency of the Delta method.

Contribution

It validates the Delta method against Bootstrap in deep learning, demonstrating comparable results with much faster computation.

Findings

Strong linear relationship between the two methods' uncertainty estimates

Delta method reduces computation time by a factor of five

Applicable to LeNet-based classifiers on MNIST and CIFAR-10 datasets

Abstract

We validate the recently introduced deep learning classification adapted Delta method by a comparison with the classical Bootstrap. We show that there is a strong linear relationship between the quantified predictive epistemic uncertainty levels obtained from the two methods when applied on two LeNet-based neural network classifiers using the MNIST and CIFAR-10 datasets. Furthermore, we demonstrate that the Delta method offers a five times computation time reduction compared to the Bootstrap.