ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring

TL;DR

ReMixMatch advances semi-supervised learning by integrating distribution alignment and augmentation anchoring, significantly reducing data requirements while maintaining high accuracy, and introduces a learned augmentation policy.

Contribution

It introduces ReMixMatch, a novel semi-supervised learning algorithm that combines distribution alignment and augmentation anchoring with learned strong augmentations.

Findings

ReMixMatch achieves high accuracy with 5-16 times less data than prior methods.

On CIFAR-10 with 250 labels, it reaches 93.73% accuracy.

The method requires only four labels per class for median accuracy.

Abstract

We improve the recently-proposed "MixMatch" semi-supervised learning algorithm by introducing two new techniques: distribution alignment and augmentation anchoring. Distribution alignment encourages the marginal distribution of predictions on unlabeled data to be close to the marginal distribution of ground-truth labels. Augmentation anchoring feeds multiple strongly augmented versions of an input into the model and encourages each output to be close to the prediction for a weakly-augmented version of the same input. To produce strong augmentations, we propose a variant of AutoAugment which learns the augmentation policy while the model is being trained. Our new algorithm, dubbed ReMixMatch, is significantly more data-efficient than prior work, requiring between $5\times$ and $16\times$ less data to reach the same accuracy. For example, on CIFAR-10 with 250 labeled examples we reach $93.73\%$ accuracy (compared to MixMatch's accuracy of $93.58\%$ with $4{,}000$ examples) and a median accuracy of $84.92\%$ with just four labels per class. We make our code and data open-source at https://github.com/google-research/remixmatch.