Active Learning by Feature Mixing

TL;DR

The paper introduces ALFA-Mix, a novel active learning method that identifies informative instances by analyzing prediction inconsistencies caused by feature interpolations, significantly reducing labeling costs especially in low-data and high-dimensional scenarios.

Contribution

ALFA-Mix is a new batch active learning approach that leverages feature mixing and prediction inconsistencies to select valuable data points, outperforming existing methods across diverse benchmarks.

Findings

Outperforms recent active learning methods in 30 settings

Significant improvements in low-data regimes and with vision transformers

Achieves state-of-the-art results in 59% and 43% of experiments for different data types

Abstract

The promise of active learning (AL) is to reduce labelling costs by selecting the most valuable examples to annotate from a pool of unlabelled data. Identifying these examples is especially challenging with high-dimensional data (e.g. images, videos) and in low-data regimes. In this paper, we propose a novel method for batch AL called ALFA-Mix. We identify unlabelled instances with sufficiently-distinct features by seeking inconsistencies in predictions resulting from interventions on their representations. We construct interpolations between representations of labelled and unlabelled instances then examine the predicted labels. We show that inconsistencies in these predictions help discovering features that the model is unable to recognise in the unlabelled instances. We derive an efficient implementation based on a closed-form solution to the optimal interpolation causing changes in predictions. Our method outperforms all recent AL approaches in 30 different settings on 12 benchmarks of images, videos, and non-visual data. The improvements are especially significant in low-data regimes and on self-trained vision transformers, where ALFA-Mix outperforms the state-of-the-art in 59% and 43% of the experiments respectively.