Minority Class Oriented Active Learning for Imbalanced Datasets

TL;DR

This paper introduces a novel active learning approach tailored for imbalanced datasets, emphasizing minority class samples to improve class representation and outperform existing methods, with insights on training schemes.

Contribution

A new active learning method for imbalanced datasets that prioritizes minority class samples and compares transfer learning-based training with traditional fine-tuning.

Findings

The proposed method outperforms baseline active learning approaches.

Transfer learning-based training scheme surpasses fine-tuning in feature transferability.

Results are validated on three imbalanced datasets.

Abstract

Active learning aims to optimize the dataset annotation process when resources are constrained. Most existing methods are designed for balanced datasets. Their practical applicability is limited by the fact that a majority of real-life datasets are actually imbalanced. Here, we introduce a new active learning method which is designed for imbalanced datasets. It favors samples likely to be in minority classes so as to reduce the imbalance of the labeled subset and create a better representation for these classes. We also compare two training schemes for active learning: (1) the one commonly deployed in deep active learning using model fine tuning for each iteration and (2) a scheme which is inspired by transfer learning and exploits generic pre-trained models and train shallow classifiers for each iteration. Evaluation is run with three imbalanced datasets. Results show that the proposed active learning method outperforms competitive baselines. Equally interesting, they also indicate that the transfer learning training scheme outperforms model fine tuning if features are transferable from the generic dataset to the unlabeled one. This last result is surprising and should encourage the community to explore the design of deep active learning methods.