Generalization Gap in Data Augmentation: Insights from Illumination

TL;DR

This paper investigates how data augmentation affects model generalization in computer vision, revealing that despite improvements, a significant gap remains due to differences in artificial and real-world visual features, especially illumination.

Contribution

It introduces the concept of visual representation variables and analyzes the impact of illumination-based data augmentation on model generalization, highlighting persistent gaps.

Findings

Data augmentation improves performance but does not eliminate the generalization gap.

Differences in artificial and natural illumination features affect model robustness.

Feature diversity in training data is crucial for better generalization.

Abstract

In the field of computer vision, data augmentation is widely used to enrich the feature complexity of training datasets with deep learning techniques. However, regarding the generalization capabilities of models, the difference in artificial features generated by data augmentation and natural visual features has not been fully revealed. This study introduces the concept of "visual representation variables" to define the possible visual variations in a task as a joint distribution of these variables. We focus on the visual representation variable "illumination", by simulating its distribution degradation and examining how data augmentation techniques enhance model performance on a classification task. Our goal is to investigate the differences in generalization between models trained with augmented data and those trained under real-world illumination conditions. Results indicate that after applying various data augmentation methods, model performance has significantly improved. Yet, a noticeable generalization gap still exists after utilizing various data augmentation methods, emphasizing the critical role of feature diversity in the training set for enhancing model generalization.