CharDiff-LP: A Diffusion Model with Character-Level Guidance for License Plate Image Restoration

TL;DR

CharDiff-LP introduces a diffusion model with character-level guidance and a novel attention mechanism to effectively restore and recognize severely degraded license plate images, significantly improving accuracy and visual quality.

Contribution

It presents a novel diffusion-based framework with character-level priors and a region-wise masking attention module for license plate image restoration.

Findings

Achieved a 28.3% reduction in character error rate on Roboflow-LP dataset.

Outperformed baseline models in restoration quality and recognition accuracy.

Demonstrated effectiveness on severely degraded license plate images.

Abstract

License plate image restoration is important not only as a preprocessing step for license plate recognition but also for enhancing evidential value, improving visual clarity, and enabling broader reuse of license plate images. We propose a novel diffusion-based framework with character-level guidance, CharDiff-LP, which effectively restores and recognizes severely degraded license plate images captured under realistic conditions. CharDiff-LP leverages fine-grained character-level priors extracted through external segmentation and Optical Character Recognition (OCR) modules tailored for low-quality license plate images. For precise and focused guidance, CharDiff-LP incorporates a novel Character-guided Attention through Region-wise Masking (CHARM) module, which ensures that each character's guidance is restricted to its own region, thereby avoiding interference with other regions. In experiments, CharDiff-LP significantly outperformed baseline restoration models in both restoration quality and recognition accuracy, achieving a 28.3% relative reduction in character error rate (CER) on the Roboflow-LP dataset compared with the best-performing baseline.