A Robust Attentional Framework for License Plate Recognition in the Wild

TL;DR

This paper introduces a robust license plate recognition framework in natural scenes, combining a CycleGAN for data augmentation and an attention-based recognition network, achieving state-of-the-art results across multiple datasets.

Contribution

The work presents a novel combination of CycleGAN-based data generation and an attention-driven recognition model for improved accuracy in unconstrained environments.

Findings

Achieved state-of-the-art performance on four public datasets.

Generated diverse training data with CycleGAN to improve robustness.

Released a new comprehensive license plate dataset from China.

Abstract

Recognizing car license plates in natural scene images is an important yet still challenging task in realistic applications. Many existing approaches perform well for license plates collected under constrained conditions, eg, shooting in frontal and horizontal view-angles and under good lighting conditions. However, their performance drops significantly in an unconstrained environment that features rotation, distortion, occlusion, blurring, shading or extreme dark or bright conditions. In this work, we propose a robust framework for license plate recognition in the wild. It is composed of a tailored CycleGAN model for license plate image generation and an elaborate designed image-to-sequence network for plate recognition. On one hand, the CycleGAN based plate generation engine alleviates the exhausting human annotation work. Massive amount of training data can be obtained with a more balanced character distribution and various shooting conditions, which helps to boost the recognition accuracy to a large extent. On the other hand, the 2D attentional based license plate recognizer with an Xception-based CNN encoder is capable of recognizing license plates with different patterns under various scenarios accurately and robustly. Without using any heuristics rule or post-processing, our method achieves the state-of-the-art performance on four public datasets, which demonstrates the generality and robustness of our framework. Moreover, we released a new license plate dataset, named "CLPD", with 1200 images from all 31 provinces in mainland China. The dataset can be available from: https://github.com/wangpengnorman/CLPD_dataset.