Incremental Self-training for Semi-supervised Learning

TL;DR

This paper introduces Incremental Self-training (IST), a semi-supervised learning method that processes unlabeled data in batches with high certainty pseudo-labels, improving accuracy and speed over traditional self-training.

Contribution

The paper proposes IST, a novel batch-wise, priority-based self-training approach that reduces time consumption and enhances performance in semi-supervised learning.

Findings

Outperforms state-of-the-art on three image classification tasks.

Improves recognition accuracy across five datasets.

Reduces training time compared to traditional self-training.

Abstract

Semi-supervised learning provides a solution to reduce the dependency of machine learning on labeled data. As one of the efficient semi-supervised techniques, self-training (ST) has received increasing attention. Several advancements have emerged to address challenges associated with noisy pseudo-labels. Previous works on self-training acknowledge the importance of unlabeled data but have not delved into their efficient utilization, nor have they paid attention to the problem of high time consumption caused by iterative learning. This paper proposes Incremental Self-training (IST) for semi-supervised learning to fill these gaps. Unlike ST, which processes all data indiscriminately, IST processes data in batches and priority assigns pseudo-labels to unlabeled samples with high certainty. Then, it processes the data around the decision boundary after the model is stabilized, enhancing classifier performance. Our IST is simple yet effective and fits existing self-training-based semi-supervised learning methods. We verify the proposed IST on five datasets and two types of backbone, effectively improving the recognition accuracy and learning speed. Significantly, it outperforms state-of-the-art competitors on three challenging image classification tasks.