Text Recognition in Real Scenarios with a Few Labeled Samples

TL;DR

This paper introduces a few-shot adversarial sequence domain adaptation method for scene text recognition, enabling high accuracy with limited labeled target samples by aligning synthetic and real data at the character level.

Contribution

The proposed FASDA approach effectively adapts synthetic models to real-world scenarios with minimal labeled data, outperforming finetuning and matching state-of-the-art methods.

Findings

Significantly outperforms finetuning schemes.

Achieves comparable performance to state-of-the-art STR methods.

Effective in scenarios with scarce labeled data.

Abstract

Scene text recognition (STR) is still a hot research topic in computer vision field due to its various applications. Existing works mainly focus on learning a general model with a huge number of synthetic text images to recognize unconstrained scene texts, and have achieved substantial progress. However, these methods are not quite applicable in many real-world scenarios where 1) high recognition accuracy is required, while 2) labeled samples are lacked. To tackle this challenging problem, this paper proposes a few-shot adversarial sequence domain adaptation (FASDA) approach to build sequence adaptation between the synthetic source domain (with many synthetic labeled samples) and a specific target domain (with only some or a few real labeled samples). This is done by simultaneously learning each character's feature representation with an attention mechanism and establishing the corresponding character-level latent subspace with adversarial learning. Our approach can maximize the character-level confusion between the source domain and the target domain, thus achieves the sequence-level adaptation with even a small number of labeled samples in the target domain. Extensive experiments on various datasets show that our method significantly outperforms the finetuning scheme, and obtains comparable performance to the state-of-the-art STR methods.