On Rendering Synthetic Images for Training an Object Detector

TL;DR

This paper introduces a method for synthesizing training images for object detection by estimating rendering parameters from few real images, improving detection performance by generating feature-similar synthetic images rather than visually realistic ones.

Contribution

The paper presents a novel approach to generate synthetic training images that match real image features, enhancing object detector training with limited real data.

Findings

Synthetic images improve detection accuracy over perturbations of real images.

Feature-based synthesis outperforms realistic-looking image generation.

Method is effective for drone, plane, and car detection tasks.

Abstract

We propose a novel approach to synthesizing images that are effective for training object detectors. Starting from a small set of real images, our algorithm estimates the rendering parameters required to synthesize similar images given a coarse 3D model of the target object. These parameters can then be reused to generate an unlimited number of training images of the object of interest in arbitrary 3D poses, which can then be used to increase classification performances. A key insight of our approach is that the synthetically generated images should be similar to real images, not in terms of image quality, but rather in terms of features used during the detector training. We show in the context of drone, plane, and car detection that using such synthetically generated images yields significantly better performances than simply perturbing real images or even synthesizing images in such way that they look very realistic, as is often done when only limited amounts of training data are available.