Well-calibrated Model Uncertainty with Temperature Scaling for Dropout Variational Inference

TL;DR

This paper extends temperature scaling to dropout variational inference to improve the calibration of model uncertainty, leading to more reliable uncertainty estimates without affecting accuracy.

Contribution

It introduces a method to calibrate dropout-based uncertainty using temperature scaling and proposes a new metric, UCE, for measuring miscalibration.

Findings

Temperature scaling reduces uncertainty calibration error significantly.

Calibrated uncertainty improves the robustness of uncertain prediction rejection.

Method maintains model accuracy while enhancing uncertainty calibration.

Abstract

Model uncertainty obtained by variational Bayesian inference with Monte Carlo dropout is prone to miscalibration. The uncertainty does not represent the model error well. In this paper, temperature scaling is extended to dropout variational inference to calibrate model uncertainty. Expected uncertainty calibration error (UCE) is presented as a metric to measure miscalibration of uncertainty. The effectiveness of this approach is evaluated on CIFAR-10/100 for recent CNN architectures. Experimental results show, that temperature scaling considerably reduces miscalibration by means of UCE and enables robust rejection of uncertain predictions. The proposed approach can easily be derived from frequentist temperature scaling and yields well-calibrated model uncertainty. It is simple to implement and does not affect the model accuracy.