On the Cross-dataset Generalization in License Plate Recognition

TL;DR

This paper investigates the generalization ability of license plate recognition models across different datasets, revealing significant performance drops when models are tested on unseen datasets, and introduces a new diverse ALPR dataset.

Contribution

It introduces a novel experimental setup for cross-dataset evaluation and provides a new publicly available ALPR dataset with diverse vehicle types and license plates.

Findings

Traditional training protocols may overestimate model performance.

Models show significant performance drops on unseen datasets.

A new dataset enhances diversity for ALPR research.

Abstract

Automatic License Plate Recognition (ALPR) systems have shown remarkable performance on license plates (LPs) from multiple regions due to advances in deep learning and the increasing availability of datasets. The evaluation of deep ALPR systems is usually done within each dataset; therefore, it is questionable if such results are a reliable indicator of generalization ability. In this paper, we propose a traditional-split versus leave-one-dataset-out experimental setup to empirically assess the cross-dataset generalization of 12 Optical Character Recognition (OCR) models applied to LP recognition on nine publicly available datasets with a great variety in several aspects (e.g., acquisition settings, image resolution, and LP layouts). We also introduce a public dataset for end-to-end ALPR that is the first to contain images of vehicles with Mercosur LPs and the one with the highest number of motorcycle images. The experimental results shed light on the limitations of the traditional-split protocol for evaluating approaches in the ALPR context, as there are significant drops in performance for most datasets when training and testing the models in a leave-one-dataset-out fashion.