Improving Model Generalization by Agreement of Learned Representations from Data Augmentation

TL;DR

This paper introduces Agreement Maximization (AgMax), a simple yet effective regularization technique that encourages learned representations of different data augmentations to agree, thereby improving model generalization across various tasks and datasets.

Contribution

The paper proposes AgMax, a novel regularization method that leverages agreement of representations from augmented data to enhance generalization, demonstrating consistent improvements over existing augmentation techniques.

Findings

AgMax improves ImageNet classification accuracy by up to 1.5%.

AgMax enhances CIFAR and Speech Commands classification results.

AgMax outperforms other augmentation methods on PascalVOC and COCO detection tasks.

Abstract

Data augmentation reduces the generalization error by forcing a model to learn invariant representations given different transformations of the input image. In computer vision, on top of the standard image processing functions, data augmentation techniques based on regional dropout such as CutOut, MixUp, and CutMix and policy-based selection such as AutoAugment demonstrated state-of-the-art (SOTA) results. With an increasing number of data augmentation algorithms being proposed, the focus is always on optimizing the input-output mapping while not realizing that there might be an untapped value in the transformed images with the same label. We hypothesize that by forcing the representations of two transformations to agree, we can further reduce the model generalization error. We call our proposed method Agreement Maximization or simply AgMax. With this simple constraint applied during training, empirical results show that data augmentation algorithms can further improve the classification accuracy of ResNet50 on ImageNet by up to 1.5%, WideResNet40-2 on CIFAR10 by up to 0.7%, WideResNet40-2 on CIFAR100 by up to 1.6%, and LeNet5 on Speech Commands Dataset by up to 1.4%. Experimental results further show that unlike other regularization terms such as label smoothing, AgMax can take advantage of the data augmentation to consistently improve model generalization by a significant margin. On downstream tasks such as object detection and segmentation on PascalVOC and COCO, AgMax pre-trained models outperforms other data augmentation methods by as much as 1.0mAP (box) and 0.5mAP (mask). Code is available at https://github.com/roatienza/agmax.