ADT-SSL: Adaptive Dual-Threshold for Semi-Supervised Learning

TL;DR

This paper introduces ADT-SSL, an adaptive dual-threshold approach for semi-supervised learning that leverages more unlabeled data by dynamically adjusting thresholds, leading to improved classification accuracy.

Contribution

The paper proposes a novel adaptive dual-threshold method that utilizes unlabeled data more effectively in SSL by dynamically adjusting thresholds based on class-specific information.

Findings

Achieves state-of-the-art accuracy on CIFAR-10, CIFAR-100, and SVHN datasets.

Effectively leverages hard samples through adaptive thresholds.

Improves model training by utilizing more unlabeled data.

Abstract

Semi-Supervised Learning (SSL) has advanced classification tasks by inputting both labeled and unlabeled data to train a model jointly. However, existing SSL methods only consider the unlabeled data whose predictions are beyond a fixed threshold (e.g., 0.95), ignoring the valuable information from those less than 0.95. We argue that these discarded data have a large proportion and are usually of hard samples, thereby benefiting the model training. This paper proposes an Adaptive Dual-Threshold method for Semi-Supervised Learning (ADT-SSL). Except for the fixed threshold, ADT extracts another class-adaptive threshold from the labeled data to take full advantage of the unlabeled data whose predictions are less than 0.95 but more than the extracted one. Accordingly, we engage CE and $L_2$ loss functions to learn from these two types of unlabeled data, respectively. For highly similar unlabeled data, we further design a novel similar loss to make the prediction of the model consistency. Extensive experiments are conducted on benchmark datasets, including CIFAR-10, CIFAR-100, and SVHN. Experimental results show that the proposed ADT-SSL achieves state-of-the-art classification accuracy.