Posterior Uncertainty Quantification in Neural Networks using Data Augmentation

TL;DR

This paper introduces MixupMP, a data augmentation-based method for more accurate uncertainty quantification in neural networks, addressing limitations of deep ensembling by modeling a realistic predictive distribution.

Contribution

MixupMP provides a novel, practical approach to uncertainty quantification using data augmentation, improving over traditional deep ensembles and grounded in Martingale posterior theory.

Findings

MixupMP outperforms existing methods in predictive accuracy.

It offers better uncertainty estimates on image classification tasks.

Empirical results demonstrate significant improvements over Bayesian and non-Bayesian approaches.

Abstract

In this paper, we approach the problem of uncertainty quantification in deep learning through a predictive framework, which captures uncertainty in model parameters by specifying our assumptions about the predictive distribution of unseen future data. Under this view, we show that deep ensembling (Lakshminarayanan et al., 2017) is a fundamentally mis-specified model class, since it assumes that future data are supported on existing observations only -- a situation rarely encountered in practice. To address this limitation, we propose MixupMP, a method that constructs a more realistic predictive distribution using popular data augmentation techniques. MixupMP operates as a drop-in replacement for deep ensembles, where each ensemble member is trained on a random simulation from this predictive distribution. Grounded in the recently-proposed framework of Martingale posteriors (Fong et al., 2023), MixupMP returns samples from an implicitly defined Bayesian posterior. Our empirical analysis showcases that MixupMP achieves superior predictive performance and uncertainty quantification on various image classification datasets, when compared with existing Bayesian and non-Bayesian approaches.