MutexMatch: Semi-Supervised Learning with Mutex-Based Consistency Regularization

TL;DR

MutexMatch introduces a novel semi-supervised learning approach that leverages both high-confidence and low-confidence unlabeled data through mutex-based consistency regularization, significantly improving performance especially with scarce labeled data.

Contribution

The paper proposes MutexMatch, a new SSL method that effectively utilizes low-confidence samples via mutex-based regularization, reducing pseudo-label errors and enhancing accuracy.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Performs well with very limited labeled data, e.g., 20 labels on CIFAR-10.

Outperforms existing SSL methods in low-label scenarios.

Abstract

The core issue in semi-supervised learning (SSL) lies in how to effectively leverage unlabeled data, whereas most existing methods tend to put a great emphasis on the utilization of high-confidence samples yet seldom fully explore the usage of low-confidence samples. In this paper, we aim to utilize low-confidence samples in a novel way with our proposed mutex-based consistency regularization, namely MutexMatch. Specifically, the high-confidence samples are required to exactly predict "what it is" by conventional True-Positive Classifier, while the low-confidence samples are employed to achieve a simpler goal -- to predict with ease "what it is not" by True-Negative Classifier. In this sense, we not only mitigate the pseudo-labeling errors but also make full use of the low-confidence unlabeled data by consistency of dissimilarity degree. MutexMatch achieves superior performance on multiple benchmark datasets, i.e., CIFAR-10, CIFAR-100, SVHN, STL-10, mini-ImageNet and Tiny-ImageNet. More importantly, our method further shows superiority when the amount of labeled data is scarce, e.g., 92.23% accuracy with only 20 labeled data on CIFAR-10. Our code and model weights have been released at https://github.com/NJUyued/MutexMatch4SSL.