Confidence Calibration for Convolutional Neural Networks Using Structured Dropout

TL;DR

This paper investigates how structured dropout can improve confidence calibration in convolutional neural networks by promoting model diversity, with empirical results on multiple datasets and applications.

Contribution

It introduces structured dropout as a method to enhance confidence calibration in CNNs by increasing model diversity, linking dropout correlation to calibration error.

Findings

Structured dropout reduces calibration error compared to naive dropout.

Model diversity correlates with improved confidence estimates.

Structured dropout benefits Bayesian active learning applications.

Abstract

In classification applications, we often want probabilistic predictions to reflect confidence or uncertainty. Dropout, a commonly used training technique, has recently been linked to Bayesian inference, yielding an efficient way to quantify uncertainty in neural network models. However, as previously demonstrated, confidence estimates computed with a naive implementation of dropout can be poorly calibrated, particularly when using convolutional networks. In this paper, through the lens of ensemble learning, we associate calibration error with the correlation between the models sampled with dropout. Motivated by this, we explore the use of structured dropout to promote model diversity and improve confidence calibration. We use the SVHN, CIFAR-10 and CIFAR-100 datasets to empirically compare model diversity and confidence errors obtained using various dropout techniques. We also show the merit of structured dropout in a Bayesian active learning application.