Quantifying Uncertainty in Deep Learning Classification with Noise in Discrete Inputs for Risk-Based Decision Making

TL;DR

This paper introduces a mathematical framework to quantify uncertainty in deep learning models dealing with discrete input noise, enhancing risk-based decision making especially in medical applications.

Contribution

It proposes a novel framework for uncertainty quantification in DNNs with discrete input errors, addressing limitations of Bayesian methods under normality assumptions.

Findings

Framework effectively identifies risk-sensitive, misclassified cases.

Compared to Monte Carlo dropout, it better detects potential misclassifications.

Application to tuberculosis treatment outcome prediction demonstrates practical utility.

Abstract

The use of Deep Neural Network (DNN) models in risk-based decision-making has attracted extensive attention with broad applications in medical, finance, manufacturing, and quality control. To mitigate prediction-related risks in decision making, prediction confidence or uncertainty should be assessed alongside the overall performance of algorithms. Recent studies on Bayesian deep learning helps quantify prediction uncertainty arises from input noises and model parameters. However, the normality assumption of input noise in these models limits their applicability to problems involving categorical and discrete feature variables in tabular datasets. In this paper, we propose a mathematical framework to quantify prediction uncertainty for DNN models. The prediction uncertainty arises from errors in predictors that follow some known finite discrete distribution. We then conducted a case study using the framework to predict treatment outcome for tuberculosis patients during their course of treatment. The results demonstrate under a certain level of risk, we can identify risk-sensitive cases, which are prone to be misclassified due to error in predictors. Comparing to the Monte Carlo dropout method, our proposed framework is more aware of misclassification cases. Our proposed framework for uncertainty quantification in deep learning can support risk-based decision making in applications when discrete errors in predictors are present.