Uncertainty Voting Ensemble for Imbalanced Deep Regression

TL;DR

This paper introduces UVOTE, an ensemble method for imbalanced deep regression that leverages probabilistic loss and uncertainty estimates to improve performance and calibration across diverse datasets.

Contribution

UVOTE combines probabilistic deep learning with ensemble techniques, replacing traditional loss with negative log-likelihood and using uncertainty for model fusion, advancing imbalanced regression handling.

Findings

UVOTE outperforms existing methods on multiple benchmarks.

The method produces better-calibrated uncertainty estimates.

It effectively handles data imbalance in deep regression tasks.

Abstract

Data imbalance is ubiquitous when applying machine learning to real-world problems, particularly regression problems. If training data are imbalanced, the learning is dominated by the densely covered regions of the target distribution and the learned regressor tends to exhibit poor performance in sparsely covered regions. Beyond standard measures like oversampling or reweighting, there are two main approaches to handling learning from imbalanced data. For regression, recent work leverages the continuity of the distribution, while for classification, the trend has been to use ensemble methods, allowing some members to specialize in predictions for sparser regions. In our method, named UVOTE, we integrate recent advances in probabilistic deep learning with an ensemble approach for imbalanced regression. We replace traditional regression losses with negative log-likelihood, which also predicts sample-wise aleatoric uncertainty. Our experiments show that this loss function handles imbalance better. Additionally, we use the predicted aleatoric uncertainty values to fuse the predictions of different expert models in the ensemble, eliminating the need for a separate aggregation module. We compare our method with existing alternatives on multiple public benchmarks and show that UVOTE consistently outperforms the prior art, while at the same time producing better-calibrated uncertainty estimates. Our code is available at https://github.com/SherryJYC/UVOTE.