SYNAuG: Exploiting Synthetic Data for Data Imbalance Problems

TL;DR

This paper introduces SYNAuG, a method that uses synthetic data to mitigate data imbalance in training datasets, improving model fairness and performance across various tasks by combining synthetic and real data.

Contribution

SYNAuG leverages synthetic data to address data imbalance, enabling effective training and fine-tuning with minimal real data, surpassing existing methods.

Findings

Training with SYNAuG followed by fine-tuning improves performance.

Synthetic data helps mitigate domain gap issues.

Method outperforms task-specific existing approaches.

Abstract

Data imbalance in training data often leads to biased predictions from trained models, which in turn causes ethical and social issues. A straightforward solution is to carefully curate training data, but given the enormous scale of modern neural networks, this is prohibitively labor-intensive and thus impractical. Inspired by recent developments in generative models, this paper explores the potential of synthetic data to address the data imbalance problem. To be specific, our method, dubbed SYNAuG, leverages synthetic data to equalize the unbalanced distribution of training data. Our experiments demonstrate that, although a domain gap between real and synthetic data exists, training with SYNAuG followed by fine-tuning with a few real samples allows to achieve impressive performance on diverse tasks with different data imbalance issues, surpassing existing task-specific methods for the same purpose.