Decomposing Representations for Deterministic Uncertainty Estimation

TL;DR

This paper introduces a novel approach to improve uncertainty estimation in machine learning by decomposing learned representations, leading to better out-of-distribution detection and interpretability.

Contribution

It proposes a method to decompose representations and integrate uncertainties, addressing limitations of existing feature density based estimators.

Findings

Enhanced OoD detection performance

Improved interpretability of uncertainty estimates

Consistent results across different OoD settings

Abstract

Uncertainty estimation is a key component in any deployed machine learning system. One way to evaluate uncertainty estimation is using "out-of-distribution" (OoD) detection, that is, distinguishing between the training data distribution and an unseen different data distribution using uncertainty. In this work, we show that current feature density based uncertainty estimators cannot perform well consistently across different OoD detection settings. To solve this, we propose to decompose the learned representations and integrate the uncertainties estimated on them separately. Through experiments, we demonstrate that we can greatly improve the performance and the interpretability of the uncertainty estimation.