Variational Imbalanced Regression: Fair Uncertainty Quantification via Probabilistic Smoothing

TL;DR

This paper introduces VIR, a probabilistic deep learning model designed for imbalanced regression tasks, which improves both accuracy and uncertainty estimation by leveraging label-based data smoothing and probabilistic reweighting.

Contribution

VIR is a novel model that incorporates label similarity for latent representation and predicts full distributions, enhancing imbalanced regression and uncertainty quantification.

Findings

VIR outperforms existing models in accuracy on real-world datasets.

VIR provides more reliable uncertainty estimates.

VIR effectively handles data imbalance in regression tasks.

Abstract

Existing regression models tend to fall short in both accuracy and uncertainty estimation when the label distribution is imbalanced. In this paper, we propose a probabilistic deep learning model, dubbed variational imbalanced regression (VIR), which not only performs well in imbalanced regression but naturally produces reasonable uncertainty estimation as a byproduct. Different from typical variational autoencoders assuming I.I.D. representations (a data point's representation is not directly affected by other data points), our VIR borrows data with similar regression labels to compute the latent representation's variational distribution; furthermore, different from deterministic regression models producing point estimates, VIR predicts the entire normal-inverse-gamma distributions and modulates the associated conjugate distributions to impose probabilistic reweighting on the imbalanced data, thereby providing better uncertainty estimation. Experiments in several real-world datasets show that our VIR can outperform state-of-the-art imbalanced regression models in terms of both accuracy and uncertainty estimation. Code will soon be available at https://github.com/Wang-ML-Lab/variational-imbalanced-regression.