How much real data do we actually need: Analyzing object detection performance using synthetic and real data

TL;DR

This paper investigates how synthetic data can replace or supplement real data in training deep object detection models, analyzing the impact of data quantity and domain similarity on performance.

Contribution

It provides a comprehensive analysis of the effects of synthetic versus real data and offers methodological insights for training with limited real data.

Findings

Synthetic data can effectively supplement real data in training.

Domain similarity influences model performance significantly.

Limited real data can be compensated with high-quality synthetic data.

Abstract

In recent years, deep learning models have resulted in a huge amount of progress in various areas, including computer vision. By nature, the supervised training of deep models requires a large amount of data to be available. This ideal case is usually not tractable as the data annotation is a tremendously exhausting and costly task to perform. An alternative is to use synthetic data. In this paper, we take a comprehensive look into the effects of replacing real data with synthetic data. We further analyze the effects of having a limited amount of real data. We use multiple synthetic and real datasets along with a simulation tool to create large amounts of cheaply annotated synthetic data. We analyze the domain similarity of each of these datasets. We provide insights about designing a methodological procedure for training deep networks using these datasets.