Road images augmentation with synthetic traffic signs using neural networks

TL;DR

This paper introduces three GAN-based methods for embedding synthetic traffic signs into real images to improve detection and classification of rare sign classes in traffic sign recognition systems.

Contribution

It presents novel GAN-based techniques for realistic synthetic traffic sign embedding, enhancing rare class detection in traffic sign recognition.

Findings

Synthetic data improves classifier accuracy

Methods enable realistic embedding of rare signs

Enhanced detection of rare traffic signs

Abstract

Traffic sign recognition is a well-researched problem in computer vision. However, the state of the art methods works only for frequent sign classes, which are well represented in training datasets. We consider the task of rare traffic sign detection and classification. We aim to solve that problem by using synthetic training data. Such training data is obtained by embedding synthetic images of signs in the real photos. We propose three methods for making synthetic signs consistent with a scene in appearance. These methods are based on modern generative adversarial network (GAN) architectures. Our proposed methods allow realistic embedding of rare traffic sign classes that are absent in the training set. We adapt a variational autoencoder for sampling plausible locations of new traffic signs in images. We demonstrate that using a mixture of our synthetic data with real data improves the accuracy of both classifier and detector.