Single-Model Uncertainties for Deep Learning

TL;DR

This paper introduces a novel approach for estimating both aleatoric and epistemic uncertainty in deep neural networks using a single model, employing Simultaneous Quantile Regression for aleatoric uncertainty and Orthonormal Certificates for epistemic uncertainty, without retraining or ensembling.

Contribution

It presents new methods for uncertainty estimation that are computationally efficient and do not require multiple models or retraining, advancing practical deployment of deep learning models.

Findings

Achieves well-calibrated prediction intervals using SQR

Effectively detects out-of-distribution samples with OCs

Competitive performance compared to ensemble methods

Abstract

We provide single-model estimates of aleatoric and epistemic uncertainty for deep neural networks. To estimate aleatoric uncertainty, we propose Simultaneous Quantile Regression (SQR), a loss function to learn all the conditional quantiles of a given target variable. These quantiles can be used to compute well-calibrated prediction intervals. To estimate epistemic uncertainty, we propose Orthonormal Certificates (OCs), a collection of diverse non-constant functions that map all training samples to zero. These certificates map out-of-distribution examples to non-zero values, signaling epistemic uncertainty. Our uncertainty estimators are computationally attractive, as they do not require ensembling or retraining deep models, and achieve competitive performance.