A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector

TL;DR

This paper introduces a robust real-time ALPR system based on YOLO, achieving high accuracy and speed across diverse datasets and outperforming commercial solutions in challenging real-world scenarios.

Contribution

The paper presents a novel YOLO-based ALPR system with specialized CNN training and data augmentation, improving robustness and accuracy in real-world conditions.

Findings

Achieved 93.53% recognition rate on SSIG dataset

Attained 78.33% recognition rate on UFPR-ALPR dataset

Operates at 47 FPS and 35 FPS on different datasets

Abstract

Automatic License Plate Recognition (ALPR) has been a frequent topic of research due to many practical applications. However, many of the current solutions are still not robust in real-world situations, commonly depending on many constraints. This paper presents a robust and efficient ALPR system based on the state-of-the-art YOLO object detector. The Convolutional Neural Networks (CNNs) are trained and fine-tuned for each ALPR stage so that they are robust under different conditions (e.g., variations in camera, lighting, and background). Specially for character segmentation and recognition, we design a two-stage approach employing simple data augmentation tricks such as inverted License Plates (LPs) and flipped characters. The resulting ALPR approach achieved impressive results in two datasets. First, in the SSIG dataset, composed of 2,000 frames from 101 vehicle videos, our system achieved a recognition rate of 93.53% and 47 Frames Per Second (FPS), performing better than both Sighthound and OpenALPR commercial systems (89.80% and 93.03%, respectively) and considerably outperforming previous results (81.80%). Second, targeting a more realistic scenario, we introduce a larger public dataset, called UFPR-ALPR dataset, designed to ALPR. This dataset contains 150 videos and 4,500 frames captured when both camera and vehicles are moving and also contains different types of vehicles (cars, motorcycles, buses and trucks). In our proposed dataset, the trial versions of commercial systems achieved recognition rates below 70%. On the other hand, our system performed better, with recognition rate of 78.33% and 35 FPS.