LEGO: Self-Supervised Representation Learning for Scene Text Images

TL;DR

LEGO introduces a self-supervised learning approach tailored for scene text images, leveraging unlabeled real data and novel pre-text tasks to improve recognition accuracy and generalization across multiple benchmarks.

Contribution

The paper proposes LEGO, a self-supervised method with three novel pre-text tasks designed specifically for scene text images, addressing the limitations of generic self-supervised methods.

Findings

LEGO outperforms previous self-supervised methods in scene text recognition.

The pre-trained model achieves state-of-the-art or comparable results on six benchmarks.

LEGO also improves performance in other text-related tasks.

Abstract

In recent years, significant progress has been made in scene text recognition by data-driven methods. However, due to the scarcity of annotated real-world data, the training of these methods predominantly relies on synthetic data. The distribution gap between synthetic and real data constrains the further performance improvement of these methods in real-world applications. To tackle this problem, a highly promising approach is to utilize massive amounts of unlabeled real data for self-supervised training, which has been widely proven effective in many NLP and CV tasks. Nevertheless, generic self-supervised methods are unsuitable for scene text images due to their sequential nature. To address this issue, we propose a Local Explicit and Global Order-aware self-supervised representation learning method (LEGO) that accounts for the characteristics of scene text images. Inspired by the human cognitive process of learning words, which involves spelling, reading, and writing, we propose three novel pre-text tasks for LEGO to model sequential, semantic, and structural features, respectively. The entire pre-training process is optimized by using a consistent Text Knowledge Codebook. Extensive experiments validate that LEGO outperforms previous scene text self-supervised methods. The recognizer incorporated with our pre-trained model achieves superior or comparable performance compared to state-of-the-art scene text recognition methods on six benchmarks. Furthermore, we demonstrate that LEGO can achieve superior performance in other text-related tasks.