Toward Unified Fine-Grained Vehicle Classification and Automatic License Plate Recognition

TL;DR

This paper introduces UFPR-VeSV, a challenging real-world dataset for fine-grained vehicle classification and license plate recognition, and explores their integration for enhanced intelligent transportation systems.

Contribution

The paper presents a new large-scale, diverse dataset and benchmarks for FGVC and ALPR, addressing their integration and real-world challenges.

Findings

The dataset includes 24,945 images of 16,297 vehicles with detailed annotations.

Benchmarking with five deep models reveals challenges in multicolored and infrared images.

Joint FGVC and ALPR approaches show potential for improved vehicle identification.

Abstract

Extracting vehicle information from surveillance images is essential for intelligent transportation systems, enabling applications such as traffic monitoring and criminal investigations. While Automatic License Plate Recognition (ALPR) is widely used, Fine-Grained Vehicle Classification (FGVC) offers a complementary approach by identifying vehicles based on attributes such as color, make, model, and type. Although there have been advances in this field, existing studies often assume well-controlled conditions, explore limited attributes, and overlook FGVC integration with ALPR. To address these gaps, we introduce UFPR-VeSV, a dataset comprising 24,945 images of 16,297 unique vehicles with annotations for 13 colors, 26 makes, 136 models, and 14 types. Collected from the Military Police of Paran\'a (Brazil) surveillance system, the dataset captures diverse real-world conditions, including partial occlusions, nighttime infrared imaging, and varying lighting. All FGVC annotations were validated using license plate information, with text and corner annotations also being provided. A qualitative and quantitative comparison with established datasets confirmed the challenging nature of our dataset. A benchmark using five deep learning models further validated this, revealing specific challenges such as handling multicolored vehicles, infrared images, and distinguishing between vehicle models that share a common platform. Additionally, we apply two optical character recognition models to license plate recognition and explore the joint use of FGVC and ALPR. The results highlight the potential of integrating these complementary tasks for real-world applications. The UFPR-VeSV dataset is publicly available at: https://github.com/Lima001/UFPR-VeSV-Dataset.