BASIL: Balanced Active Semi-supervised Learning for Class Imbalanced Datasets

TL;DR

BASIL is a novel active semi-supervised learning algorithm that effectively balances class distribution during data selection, improving model fairness and accuracy on imbalanced datasets.

Contribution

It introduces a submodular mutual information based approach for balanced data selection in SSL, enhancing performance across various methods.

Findings

BASIL outperforms existing active learning methods in class imbalance scenarios.

The approach improves SSL model fairness and accuracy.

Effective on medical imaging datasets like Path-MNIST and Organ-MNIST.

Abstract

Current semi-supervised learning (SSL) methods assume a balance between the number of data points available for each class in both the labeled and the unlabeled data sets. However, there naturally exists a class imbalance in most real-world datasets. It is known that training models on such imbalanced datasets leads to biased models, which in turn lead to biased predictions towards the more frequent classes. This issue is further pronounced in SSL methods, as they would use this biased model to obtain psuedo-labels (on the unlabeled data) during training. In this paper, we tackle this problem by attempting to select a balanced labeled dataset for SSL that would result in an unbiased model. Unfortunately, acquiring a balanced labeled dataset from a class imbalanced distribution in one shot is challenging. We propose BASIL (Balanced Active Semi-supervIsed Learning), a novel algorithm that optimizes the submodular mutual information (SMI) functions in a per-class fashion to gradually select a balanced dataset in an active learning loop. Importantly, our technique can be efficiently used to improve the performance of any SSL method. Our experiments on Path-MNIST and Organ-MNIST medical datasets for a wide array of SSL methods show the effectiveness of Basil. Furthermore, we observe that Basil outperforms the state-of-the-art diversity and uncertainty based active learning methods since the SMI functions select a more balanced dataset.