SMILE: Sequence-to-Sequence Domain Adaption with Minimizing Latent Entropy for Text Image Recognition

TL;DR

This paper introduces SMILE, a novel unsupervised domain adaptation method for text image recognition that minimizes latent entropy in sequence-to-sequence models, effectively bridging the gap between synthetic and real-world images.

Contribution

The paper proposes a new UDA approach using latent entropy minimization and class-balanced self-paced learning tailored for sequence-to-sequence text recognition models.

Findings

Outperforms existing UDA methods on multiple benchmarks

Achieves significant improvements in recognition accuracy

Codes are publicly available for reproducibility

Abstract

Training recognition models with synthetic images have achieved remarkable results in text recognition. However, recognizing text from real-world images still faces challenges due to the domain shift between synthetic and real-world text images. One of the strategies to eliminate the domain difference without manual annotation is unsupervised domain adaptation (UDA). Due to the characteristic of sequential labeling tasks, most popular UDA methods cannot be directly applied to text recognition. To tackle this problem, we proposed a UDA method with minimizing latent entropy on sequence-to-sequence attention-based models with classbalanced self-paced learning. Our experiments show that our proposed framework achieves better recognition results than the existing methods on most UDA text recognition benchmarks. All codes are publicly available.