Data Augmentation via Structured Adversarial Perturbations

TL;DR

This paper introduces a method for generating structured adversarial examples for data augmentation, which enhances model generalization by applying natural, meaningful transformations instead of unstructured perturbations.

Contribution

The paper proposes a novel approach to create structured adversarial perturbations by projecting raw adversarial gradients onto a subspace of desired transformations, improving data augmentation quality.

Findings

Structured adversarial augmentation improves model generalization.

The method effectively generates photometric and geometric transformations.

Training on structured adversarial examples outperforms unstructured approaches.

Abstract

Data augmentation is a major component of many machine learning methods with state-of-the-art performance. Common augmentation strategies work by drawing random samples from a space of transformations. Unfortunately, such sampling approaches are limited in expressivity, as they are unable to scale to rich transformations that depend on numerous parameters due to the curse of dimensionality. Adversarial examples can be considered as an alternative scheme for data augmentation. By being trained on the most difficult modifications of the inputs, the resulting models are then hopefully able to handle other, presumably easier, modifications as well. The advantage of adversarial augmentation is that it replaces sampling with the use of a single, calculated perturbation that maximally increases the loss. The downside, however, is that these raw adversarial perturbations appear rather unstructured; applying them often does not produce a natural transformation, contrary to a desirable data augmentation technique. To address this, we propose a method to generate adversarial examples that maintain some desired natural structure. We first construct a subspace that only contains perturbations with the desired structure. We then project the raw adversarial gradient onto this space to select a structured transformation that would maximally increase the loss when applied. We demonstrate this approach through two types of image transformations: photometric and geometric. Furthermore, we show that training on such structured adversarial images improves generalization.