Striking the Right Balance with Uncertainty

TL;DR

This paper introduces a Bayesian uncertainty-based framework for addressing class imbalance in machine learning, improving model generalization by modeling samples as distributions and adjusting decision boundaries accordingly.

Contribution

It proposes a novel uncertainty-aware loss and modeling approach that extends decision boundaries and captures sample distributions, enhancing robustness on imbalanced datasets.

Findings

Significant performance gains on six benchmark datasets.

Effective modeling of samples as multivariate Gaussian distributions.

Improved generalization and robustness in imbalanced classification tasks.

Abstract

Learning unbiased models on imbalanced datasets is a significant challenge. Rare classes tend to get a concentrated representation in the classification space which hampers the generalization of learned boundaries to new test examples. In this paper, we demonstrate that the Bayesian uncertainty estimates directly correlate with the rarity of classes and the difficulty level of individual samples. Subsequently, we present a novel framework for uncertainty based class imbalance learning that follows two key insights: First, classification boundaries should be extended further away from a more uncertain (rare) class to avoid overfitting and enhance its generalization. Second, each sample should be modeled as a multi-variate Gaussian distribution with a mean vector and a covariance matrix defined by the sample's uncertainty. The learned boundaries should respect not only the individual samples but also their distribution in the feature space. Our proposed approach efficiently utilizes sample and class uncertainty information to learn robust features and more generalizable classifiers. We systematically study the class imbalance problem and derive a novel loss formulation for max-margin learning based on Bayesian uncertainty measure. The proposed method shows significant performance improvements on six benchmark datasets for face verification, attribute prediction, digit/object classification and skin lesion detection.