$\Delta$-UQ: Accurate Uncertainty Quantification via Anchor Marginalization

TL;DR

$ ext{ extDelta}$-UQ introduces a novel anchoring-based uncertainty estimation method that improves accuracy and robustness across various predictive modeling tasks by effectively capturing total uncertainty.

Contribution

The paper proposes $ ext{ extDelta}$-UQ, a new uncertainty estimator leveraging input anchoring and encoding, which outperforms existing methods in diverse applications.

Findings

$ ext{ extDelta}$-UQ achieves superior uncertainty estimation accuracy.

It effectively handles out-of-distribution detection and calibration.

The method demonstrates broad applicability across regression and classification tasks.

Abstract

We present $\Delta$-UQ -- a novel, general-purpose uncertainty estimator using the concept of anchoring in predictive models. Anchoring works by first transforming the input into a tuple consisting of an anchor point drawn from a prior distribution, and a combination of the input sample with the anchor using a pretext encoding scheme. This encoding is such that the original input can be perfectly recovered from the tuple -- regardless of the choice of the anchor. Therefore, any predictive model should be able to predict the target response from the tuple alone (since it implicitly represents the input). Moreover, by varying the anchors for a fixed sample, we can estimate uncertainty in the prediction even using only a single predictive model. We find this uncertainty is deeply connected to improper sampling of the input data, and inherent noise, enabling us to estimate the total uncertainty in any system. With extensive empirical studies on a variety of use-cases, we demonstrate that $\Delta$-UQ outperforms several competitive baselines. Specifically, we study model fitting, sequential model optimization, model based inversion in the regression setting and out of distribution detection, & calibration under distribution shifts for classification.