A Simple Framework to Quantify Different Types of Uncertainty in Deep Neural Networks for Image Classification

TL;DR

This paper introduces a comprehensive framework for quantifying three types of uncertainty in deep neural networks for image classification, enhancing safety and reliability in high-stakes applications.

Contribution

It presents a novel, unified approach combining ensemble methods, auto-encoders, and activation outputs to measure model, distributional, and data uncertainties.

Findings

Effective in capturing different uncertainty types

Improves decision-making in high-stakes scenarios

Demonstrates strong performance on standard datasets

Abstract

Quantifying uncertainty in a model's predictions is important as it enables the safety of an AI system to be increased by acting on the model's output in an informed manner. This is crucial for applications where the cost of an error is high, such as in autonomous vehicle control, medical image analysis, financial estimations or legal fields. Deep Neural Networks are powerful predictors that have recently achieved state-of-the-art performance on a wide spectrum of tasks. Quantifying predictive uncertainty in DNNs is a challenging and yet on-going problem. In this paper we propose a complete framework to capture and quantify three known types of uncertainty in DNNs for the task of image classification. This framework includes an ensemble of CNNs for model uncertainty, a supervised reconstruction auto-encoder to capture distributional uncertainty and using the output of activation functions in the last layer of the network, to capture data uncertainty. Finally we demonstrate the efficiency of our method on popular image datasets for classification.