Learning to Generate Synthetic Data via Compositing

TL;DR

This paper introduces a task-aware, adversarial synthetic data generation framework that improves training of classifiers and detectors by producing realistic, meaningful training samples, leading to better performance with less data.

Contribution

The paper proposes a novel adversarial training approach for synthetic data generation that incorporates a trainable synthesizer and discriminator, improving model performance across multiple tasks.

Findings

Outperforms baseline on AffNIST with half the data

Improves object detection accuracy on VOC and GMU datasets

Achieves state-of-the-art results in synthetic data augmentation

Abstract

We present a task-aware approach to synthetic data generation. Our framework employs a trainable synthesizer network that is optimized to produce meaningful training samples by assessing the strengths and weaknesses of a `target' network. The synthesizer and target networks are trained in an adversarial manner wherein each network is updated with a goal to outdo the other. Additionally, we ensure the synthesizer generates realistic data by pairing it with a discriminator trained on real-world images. Further, to make the target classifier invariant to blending artefacts, we introduce these artefacts to background regions of the training images so the target does not over-fit to them. We demonstrate the efficacy of our approach by applying it to different target networks including a classification network on AffNIST, and two object detection networks (SSD, Faster-RCNN) on different datasets. On the AffNIST benchmark, our approach is able to surpass the baseline results with just half the training examples. On the VOC person detection benchmark, we show improvements of up to 2.7% as a result of our data augmentation. Similarly on the GMU detection benchmark, we report a performance boost of 3.5% in mAP over the baseline method, outperforming the previous state of the art approaches by up to 7.5% on specific categories.