Auto-Annotation Quality Prediction for Semi-Supervised Learning with Ensembles

TL;DR

This paper introduces a method to improve semi-supervised learning by filtering auto-annotations using ensemble consensus, significantly reducing the need for manual labels in semantic segmentation tasks.

Contribution

It proposes a novel auto-annotation filtering technique based on ensemble consensus to enhance semi-supervised learning performance.

Findings

Filtering auto-annotations improves model accuracy.

Achieves state-of-the-art performance with only 30% manual labels.

Auto-annotation filtering reduces reliance on extensive manual labeling.

Abstract

Auto-annotation by ensemble of models is an efficient method of learning on unlabeled data. Wrong or inaccurate annotations generated by the ensemble may lead to performance degradation of the trained model. To deal with this problem we propose filtering the auto-labeled data using a trained model that predicts the quality of the annotation from the degree of consensus between ensemble models. Using semantic segmentation as an example, we show the advantage of the proposed auto-annotation filtering over training on data contaminated with inaccurate labels. Moreover, our experimental results show that in the case of semantic segmentation, the performance of a state-of-the-art model can be achieved by training it with only a fraction (30$\%$) of the original manually labeled data set, and replacing the rest with the auto-annotated, quality filtered labels.