Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization

TL;DR

This paper introduces a method for training deep neural networks for object detection using synthetic data with domain randomization, reducing reliance on real data and improving performance after fine-tuning.

Contribution

The study demonstrates that non-realistic synthetic data with domain randomization can effectively train neural networks, which are further improved with minimal real data fine-tuning.

Findings

Synthetic data with domain randomization achieves compelling detection performance.

Fine-tuning on real data enhances the network beyond real-data-only training.

The approach reduces the need for large annotated real-world datasets.

Abstract

We present a system for training deep neural networks for object detection using synthetic images. To handle the variability in real-world data, the system relies upon the technique of domain randomization, in which the parameters of the simulator$-$such as lighting, pose, object textures, etc.$-$are randomized in non-realistic ways to force the neural network to learn the essential features of the object of interest. We explore the importance of these parameters, showing that it is possible to produce a network with compelling performance using only non-artistically-generated synthetic data. With additional fine-tuning on real data, the network yields better performance than using real data alone. This result opens up the possibility of using inexpensive synthetic data for training neural networks while avoiding the need to collect large amounts of hand-annotated real-world data or to generate high-fidelity synthetic worlds$-$both of which remain bottlenecks for many applications. The approach is evaluated on bounding box detection of cars on the KITTI dataset.