RenderGAN: Generating Realistic Labeled Data

TL;DR

RenderGAN is a novel framework that combines 3D modeling and GANs to generate realistic, labeled images, significantly reducing data annotation costs for training deep neural networks in computer vision tasks.

Contribution

We introduce RenderGAN, a new method that synthesizes realistic labeled images by integrating 3D models with GANs, improving training data quality and quantity.

Findings

Generated images are highly realistic and label-preserving.

Training DCNNs on RenderGAN data outperforms baseline methods.

RenderGAN reduces the need for manual data annotation.

Abstract

Deep Convolutional Neuronal Networks (DCNNs) are showing remarkable performance on many computer vision tasks. Due to their large parameter space, they require many labeled samples when trained in a supervised setting. The costs of annotating data manually can render the use of DCNNs infeasible. We present a novel framework called RenderGAN that can generate large amounts of realistic, labeled images by combining a 3D model and the Generative Adversarial Network framework. In our approach, image augmentations (e.g. lighting, background, and detail) are learned from unlabeled data such that the generated images are strikingly realistic while preserving the labels known from the 3D model. We apply the RenderGAN framework to generate images of barcode-like markers that are attached to honeybees. Training a DCNN on data generated by the RenderGAN yields considerably better performance than training it on various baselines.