Pruning the Unlabeled Data to Improve Semi-Supervised Learning

TL;DR

This paper introduces PruneSSL, a method that improves semi-supervised learning by selectively removing less separable unlabeled data, leading to better performance despite using less data.

Contribution

PruneSSL is a novel data pruning technique that enhances SSL by increasing data separability, resulting in state-of-the-art image classification performance.

Findings

PruneSSL improves SSL accuracy across multiple datasets.

Selective data removal enhances model performance.

State-of-the-art results achieved with less data.

Abstract

In the domain of semi-supervised learning (SSL), the conventional approach involves training a learner with a limited amount of labeled data alongside a substantial volume of unlabeled data, both drawn from the same underlying distribution. However, for deep learning models, this standard practice may not yield optimal results. In this research, we propose an alternative perspective, suggesting that distributions that are more readily separable could offer superior benefits to the learner as compared to the original distribution. To achieve this, we present PruneSSL, a practical technique for selectively removing examples from the original unlabeled dataset to enhance its separability. We present an empirical study, showing that although PruneSSL reduces the quantity of available training data for the learner, it significantly improves the performance of various competitive SSL algorithms, thereby achieving state-of-the-art results across several image classification tasks.