Adversarial Generation of Training Examples: Applications to Moving Vehicle License Plate Recognition

TL;DR

This paper demonstrates that synthetic training data generated by GANs can significantly improve vehicle license plate recognition accuracy, especially on mobile devices, by integrating deep learning models with a novel data augmentation pipeline.

Contribution

It introduces a new pipeline for generating and utilizing GAN-based synthetic images to enhance license plate recognition accuracy, including a lightweight convolutional RNN for mobile deployment.

Findings

GAN-generated images improve recognition accuracy by 7.5%.

The lightweight convolutional RNN is half the size and twice as fast on CPU.

The framework achieves competitive accuracy on challenging datasets.

Abstract

Generative Adversarial Networks (GAN) have attracted much research attention recently, leading to impressive results for natural image generation. However, to date little success was observed in using GAN generated images for improving classification tasks. Here we attempt to explore, in the context of car license plate recognition, whether it is possible to generate synthetic training data using GAN to improve recognition accuracy. With a carefully-designed pipeline, we show that the answer is affirmative. First, a large-scale image set is generated using the generator of GAN, without manual annotation. Then, these images are fed to a deep convolutional neural network (DCNN) followed by a bidirectional recurrent neural network (BRNN) with long short-term memory (LSTM), which performs the feature learning and sequence labelling. Finally, the pre-trained model is fine-tuned on real images. Our experimental results on a few data sets demonstrate the effectiveness of using GAN images: an improvement of 7.5% over a strong baseline with moderate-sized real data being available. We show that the proposed framework achieves competitive recognition accuracy on challenging test datasets. We also leverage the depthwise separate convolution to construct a lightweight convolutional RNN, which is about half size and 2x faster on CPU. Combining this framework and the proposed pipeline, we make progress in performing accurate recognition on mobile and embedded devices.