Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data

TL;DR

This paper introduces FullMatch, a novel semi-supervised learning framework that leverages all unlabeled data through Entropy Meaning Loss and Adaptive Negative Learning, significantly improving performance over existing methods.

Contribution

The paper proposes two new techniques, EML and ANL, to better utilize all unlabeled data in SSL, integrated into FixMatch to create the effective FullMatch framework.

Findings

FullMatch outperforms FixMatch on multiple SSL benchmarks.

FullMatch achieves state-of-the-art results when combined with FlexMatch.

The proposed methods do not require additional parameters or hyperparameters.

Abstract

Semi-supervised learning (SSL) has attracted enormous attention due to its vast potential of mitigating the dependence on large labeled datasets. The latest methods (e.g., FixMatch) use a combination of consistency regularization and pseudo-labeling to achieve remarkable successes. However, these methods all suffer from the waste of complicated examples since all pseudo-labels have to be selected by a high threshold to filter out noisy ones. Hence, the examples with ambiguous predictions will not contribute to the training phase. For better leveraging all unlabeled examples, we propose two novel techniques: Entropy Meaning Loss (EML) and Adaptive Negative Learning (ANL). EML incorporates the prediction distribution of non-target classes into the optimization objective to avoid competition with target class, and thus generating more high-confidence predictions for selecting pseudo-label. ANL introduces the additional negative pseudo-label for all unlabeled data to leverage low-confidence examples. It adaptively allocates this label by dynamically evaluating the top-k performance of the model. EML and ANL do not introduce any additional parameter and hyperparameter. We integrate these techniques with FixMatch, and develop a simple yet powerful framework called FullMatch. Extensive experiments on several common SSL benchmarks (CIFAR-10/100, SVHN, STL-10 and ImageNet) demonstrate that FullMatch exceeds FixMatch by a large margin. Integrated with FlexMatch (an advanced FixMatch-based framework), we achieve state-of-the-art performance. Source code is at https://github.com/megvii-research/FullMatch.