Training Ensembles with Inliers and Outliers for Semi-supervised Active Learning

TL;DR

This paper introduces a novel semi-supervised active learning approach that combines joint classifier training, ensembling, and pseudo-labeling to effectively handle outliers, improving accuracy without explicit outlier detection.

Contribution

The work demonstrates that integrating ensembling and joint training with pseudo-labeling enhances active learning performance in the presence of outliers, eliminating the need for explicit outlier detection.

Findings

Ensembling improves pseudo-label accuracy.

Joint training boosts classifier performance with unlabeled data.

The approach outperforms existing methods in active learning scenarios.

Abstract

Deep active learning in the presence of outlier examples poses a realistic yet challenging scenario. Acquiring unlabeled data for annotation requires a delicate balance between avoiding outliers to conserve the annotation budget and prioritizing useful inlier examples for effective training. In this work, we present an approach that leverages three highly synergistic components, which are identified as key ingredients: joint classifier training with inliers and outliers, semi-supervised learning through pseudo-labeling, and model ensembling. Our work demonstrates that ensembling significantly enhances the accuracy of pseudo-labeling and improves the quality of data acquisition. By enabling semi-supervision through the joint training process, where outliers are properly handled, we observe a substantial boost in classifier accuracy through the use of all available unlabeled examples. Notably, we reveal that the integration of joint training renders explicit outlier detection unnecessary; a conventional component for acquisition in prior work. The three key components align seamlessly with numerous existing approaches. Through empirical evaluations, we showcase that their combined use leads to a performance increase. Remarkably, despite its simplicity, our proposed approach outperforms all other methods in terms of performance. Code: https://github.com/vladan-stojnic/active-outliers