Exploring the Equivalence of Closed-Set Generative and Real Data Augmentation in Image Classification

TL;DR

This paper investigates whether synthetic images generated by advanced models can effectively replace or supplement real data for image classification, providing empirical guidelines for data augmentation strategies.

Contribution

It systematically compares real and synthetic data augmentation, quantifies their equivalence, and offers practical guidelines for using synthetic data in classification tasks.

Findings

Synthetic data can match real data augmentation performance when scaled appropriately.

Quantitative measures of equivalence between real and synthetic data augmentation are established.

The effectiveness of synthetic augmentation varies with dataset size and domain.

Abstract

In this paper, we address a key scientific problem in machine learning: Given a training set for an image classification task, can we train a generative model on this dataset to enhance the classification performance? (i.e., closed-set generative data augmentation). We start by exploring the distinctions and similarities between real images and closed-set synthetic images generated by advanced generative models. Through extensive experiments, we offer systematic insights into the effective use of closed-set synthetic data for augmentation. Notably, we empirically determine the equivalent scale of synthetic images needed for augmentation. In addition, we also show quantitative equivalence between the real data augmentation and open-set generative augmentation (generative models trained using data beyond the given training set). While it aligns with the common intuition that real images are generally preferred, our empirical formulation also offers a guideline to quantify the increased scale of synthetic data augmentation required to achieve comparable image classification performance. Our results on natural and medical image datasets further illustrate how this effect varies with the baseline training set size and the amount of synthetic data incorporated.