Swapping Semantic Contents for Mixing Images

TL;DR

SciMix introduces a novel data augmentation method that embeds semantic style into image backgrounds, generating mixed samples that enhance semi-supervised and supervised learning performance with limited labeled data.

Contribution

The paper presents SciMix, a framework that learns to embed semantic styles into images for improved data augmentation in low-label scenarios.

Findings

SciMix generates mixed samples inheriting characteristics from parent images.

Using SciMix improves semi-supervised learning frameworks like Mean Teacher and FixMatch.

SciMix enhances fully supervised learning on small labeled datasets.

Abstract

Deep architecture have proven capable of solving many tasks provided a sufficient amount of labeled data. In fact, the amount of available labeled data has become the principal bottleneck in low label settings such as Semi-Supervised Learning. Mixing Data Augmentations do not typically yield new labeled samples, as indiscriminately mixing contents creates between-class samples. In this work, we introduce the SciMix framework that can learn to generator to embed a semantic style code into image backgrounds, we obtain new mixing scheme for data augmentation. We then demonstrate that SciMix yields novel mixed samples that inherit many characteristics from their non-semantic parents. Afterwards, we verify those samples can be used to improve the performance semi-supervised frameworks like Mean Teacher or Fixmatch, and even fully supervised learning on a small labeled dataset.