Fixing Overconfidence in Dynamic Neural Networks

TL;DR

This paper introduces a post-hoc uncertainty quantification method for dynamic neural networks, improving their ability to distinguish between hard and easy samples, thereby enhancing accuracy and calibration across multiple datasets.

Contribution

It proposes a computationally efficient probabilistic approach to better quantify uncertainty in dynamic neural networks, addressing a key challenge in their deployment.

Findings

Improved accuracy on CIFAR-100, ImageNet, and Caltech-256.

Enhanced uncertainty capture and calibration.

Better decision-making for computational budget allocation.

Abstract

Dynamic neural networks are a recent technique that promises a remedy for the increasing size of modern deep learning models by dynamically adapting their computational cost to the difficulty of the inputs. In this way, the model can adjust to a limited computational budget. However, the poor quality of uncertainty estimates in deep learning models makes it difficult to distinguish between hard and easy samples. To address this challenge, we present a computationally efficient approach for post-hoc uncertainty quantification in dynamic neural networks. We show that adequately quantifying and accounting for both aleatoric and epistemic uncertainty through a probabilistic treatment of the last layers improves the predictive performance and aids decision-making when determining the computational budget. In the experiments, we show improvements on CIFAR-100, ImageNet, and Caltech-256 in terms of accuracy, capturing uncertainty, and calibration error.