Adversarial AutoAugment

TL;DR

This paper introduces Adversarial AutoAugment, a computationally efficient method that uses adversarial training to learn data augmentation policies, significantly improving neural network generalization with reduced computational cost.

Contribution

It proposes an adversarial approach to automatically learn augmentation policies that are more effective and computationally affordable than prior methods like AutoAugment.

Findings

Achieves state-of-the-art results on CIFAR-10 with 1.36% error.

Replaces AutoAugment with 12x less computation and 11x faster training.

Improves ImageNet top-1 accuracy to 79.40% with ResNet-50.

Abstract

Data augmentation (DA) has been widely utilized to improve generalization in training deep neural networks. Recently, human-designed data augmentation has been gradually replaced by automatically learned augmentation policy. Through finding the best policy in well-designed search space of data augmentation, AutoAugment can significantly improve validation accuracy on image classification tasks. However, this approach is not computationally practical for large-scale problems. In this paper, we develop an adversarial method to arrive at a computationally-affordable solution called Adversarial AutoAugment, which can simultaneously optimize target related object and augmentation policy search loss. The augmentation policy network attempts to increase the training loss of a target network through generating adversarial augmentation policies, while the target network can learn more robust features from harder examples to improve the generalization. In contrast to prior work, we reuse the computation in target network training for policy evaluation, and dispense with the retraining of the target network. Compared to AutoAugment, this leads to about 12x reduction in computing cost and 11x shortening in time overhead on ImageNet. We show experimental results of our approach on CIFAR-10/CIFAR-100, ImageNet, and demonstrate significant performance improvements over state-of-the-art. On CIFAR-10, we achieve a top-1 test error of 1.36%, which is the currently best performing single model. On ImageNet, we achieve a leading performance of top-1 accuracy 79.40% on ResNet-50 and 80.00% on ResNet-50-D without extra data.