RegMixMatch: Optimizing Mixup Utilization in Semi-Supervised Learning

TL;DR

RegMixMatch is a novel semi-supervised learning framework that optimizes Mixup usage, including low-confidence samples, to improve label purity and overall performance, achieving state-of-the-art results.

Contribution

It introduces semi-supervised RegMixup and class-aware Mixup techniques to better utilize all unlabeled data and reduce confirmation bias in SSL.

Findings

Achieves state-of-the-art results on SSL benchmarks.

Effectively utilizes low-confidence samples with class-aware Mixup.

Improves label purity and reduces confirmation bias.

Abstract

Consistency regularization and pseudo-labeling have significantly advanced semi-supervised learning (SSL). Prior works have effectively employed Mixup for consistency regularization in SSL. However, our findings indicate that applying Mixup for consistency regularization may degrade SSL performance by compromising the purity of artificial labels. Moreover, most pseudo-labeling based methods utilize thresholding strategy to exclude low-confidence data, aiming to mitigate confirmation bias; however, this approach limits the utility of unlabeled samples. To address these challenges, we propose RegMixMatch, a novel framework that optimizes the use of Mixup with both high- and low-confidence samples in SSL. First, we introduce semi-supervised RegMixup, which effectively addresses reduced artificial labels purity by using both mixed samples and clean samples for training. Second, we develop a class-aware Mixup technique that integrates information from the top-2 predicted classes into low-confidence samples and their artificial labels, reducing the confirmation bias associated with these samples and enhancing their effective utilization. Experimental results demonstrate that RegMixMatch achieves state-of-the-art performance across various SSL benchmarks.