Visual Data Augmentation through Learning

TL;DR

This paper introduces a novel data augmentation method that learns realistic local transformations in images, enabling the synthesis of new, plausible images to improve machine learning models.

Contribution

The paper proposes a learning-based augmentation technique that models local transformations in images, producing more natural augmented data compared to traditional methods.

Findings

Generated images are realistic and diverse.

The method improves model invariance and robustness.

Qualitative and quantitative results validate effectiveness.

Abstract

The rapid progress in machine learning methods has been empowered by i) huge datasets that have been collected and annotated, ii) improved engineering (e.g. data pre-processing/normalization). The existing datasets typically include several million samples, which constitutes their extension a colossal task. In addition, the state-of-the-art data-driven methods demand a vast amount of data, hence a standard engineering trick employed is artificial data augmentation for instance by adding into the data cropped and (affinely) transformed images. However, this approach does not correspond to any change in the natural 3D scene. We propose instead to perform data augmentation through learning realistic local transformations. We learn a forward and an inverse transformation that maps an image from the high-dimensional space of pixel intensities to a latent space which varies (approximately) linearly with the latent space of a realistically transformed version of the image. Such transformed images can be considered two successive frames in a video. Next, we utilize these transformations to learn a linear model that modifies the latent spaces and then use the inverse transformation to synthesize a new image. We argue that the this procedure produces powerful invariant representations. We perform both qualitative and quantitative experiments that demonstrate our proposed method creates new realistic images.