Wasserstein Dropout

TL;DR

Wasserstein Dropout is a novel non-parametric method for uncertainty quantification in neural network regression that outperforms existing approaches, especially under distributional shifts.

Contribution

It introduces a new Wasserstein distance-based objective for dropout, improving uncertainty estimates without relying on parametric assumptions.

Findings

Outperforms state-of-the-art methods in accuracy and stability

Effective under distributional shift

Provides more reliable uncertainty estimates

Abstract

Despite of its importance for safe machine learning, uncertainty quantification for neural networks is far from being solved. State-of-the-art approaches to estimate neural uncertainties are often hybrid, combining parametric models with explicit or implicit (dropout-based) ensembling. We take another pathway and propose a novel approach to uncertainty quantification for regression tasks, Wasserstein dropout, that is purely non-parametric. Technically, it captures aleatoric uncertainty by means of dropout-based sub-network distributions. This is accomplished by a new objective which minimizes the Wasserstein distance between the label distribution and the model distribution. An extensive empirical analysis shows that Wasserstein dropout outperforms state-of-the-art methods, on vanilla test data as well as under distributional shift, in terms of producing more accurate and stable uncertainty estimates.