Epsilon Consistent Mixup: Structural Regularization with an Adaptive Consistency-Interpolation Tradeoff

TL;DR

This paper introduces $psilon$-Consistent Mixup ($psilon$mu), a novel regularization method that adaptively combines Mixup interpolation with consistency regularization, improving semi-supervised classification especially with limited labels.

Contribution

The paper proposes $psilon$mu, a new adaptive regularization technique that enhances Mixup by integrating consistency regularization, leading to better semi-supervised learning performance.

Findings

Improves semi-supervised classification accuracy on SVHN and CIFAR10.

Produces more accurate synthetic labels than Mixup.

Yields larger gains in low label-availability scenarios.

Abstract

In this paper we propose $\epsilon$-Consistent Mixup ($\epsilon$mu). $\epsilon$mu is a data-based structural regularization technique that combines Mixup's linear interpolation with consistency regularization in the Mixup direction, by compelling a simple adaptive tradeoff between the two. This learnable combination of consistency and interpolation induces a more flexible structure on the evolution of the response across the feature space and is shown to improve semi-supervised classification accuracy on the SVHN and CIFAR10 benchmark datasets, yielding the largest gains in the most challenging low label-availability scenarios. Empirical studies comparing $\epsilon$mu and Mixup are presented and provide insight into the mechanisms behind $\epsilon$mu's effectiveness. In particular, $\epsilon$mu is found to produce more accurate synthetic labels and more confident predictions than Mixup.