Reliable Label Bootstrapping for Semi-Supervised Learning

TL;DR

ReLaB is an unsupervised pre-processing method that enhances semi-supervised learning by propagating labels through self-supervised features and selecting reliable samples, significantly reducing the need for labeled data.

Contribution

The paper introduces ReLaB, a novel label bootstrapping method that improves semi-supervised learning in low-label scenarios by combining self-supervised features, label propagation, and noise detection.

Findings

ReLaB achieves lower error rates on CIFAR-10, CIFAR-100, and mini-ImageNet.

Effective label propagation depends on network architecture and self-supervised algorithms.

ReLaB significantly reduces the number of labeled samples needed for competitive performance.

Abstract

Reducing the amount of labels required to train convolutional neural networks without performance degradation is key to effectively reduce human annotation efforts. We propose Reliable Label Bootstrapping (ReLaB), an unsupervised preprossessing algorithm which improves the performance of semi-supervised algorithms in extremely low supervision settings. Given a dataset with few labeled samples, we first learn meaningful self-supervised, latent features for the data. Second, a label propagation algorithm propagates the known labels on the unsupervised features, effectively labeling the full dataset in an automatic fashion. Third, we select a subset of correctly labeled (reliable) samples using a label noise detection algorithm. Finally, we train a semi-supervised algorithm on the extended subset. We show that the selection of the network architecture and the self-supervised algorithm are important factors to achieve successful label propagation and demonstrate that ReLaB substantially improves semi-supervised learning in scenarios of very limited supervision on CIFAR-10, CIFAR-100 and mini-ImageNet. We reach average error rates of $\boldsymbol{22.34}$ with 1 random labeled sample per class on CIFAR-10 and lower this error to $\boldsymbol{8.46}$ when the labeled sample in each class is highly representative. Our work is fully reproducible: https://github.com/PaulAlbert31/ReLaB.