Harnessing the Power of Multi-Lingual Datasets for Pre-training: Towards Enhancing Text Spotting Performance

TL;DR

This paper explores multi-lingual datasets and intermediate feature representations to improve domain adaptation in scene text spotting, demonstrating significant accuracy and efficiency gains across diverse benchmarks.

Contribution

It introduces a domain-adaptive training approach using multi-domain data and evaluates a transformer-based model, Swin-TESTR, for improved scene text spotting.

Findings

Intermediate representations enhance performance across domains.

Multi-lingual and multi-domain training improves adaptability.

Swin-TESTR achieves state-of-the-art results in accuracy and efficiency.

Abstract

The adaptation capability to a wide range of domains is crucial for scene text spotting models when deployed to real-world conditions. However, existing state-of-the-art (SOTA) approaches usually incorporate scene text detection and recognition simply by pretraining on natural scene text datasets, which do not directly exploit the intermediate feature representations between multiple domains. Here, we investigate the problem of domain-adaptive scene text spotting, i.e., training a model on multi-domain source data such that it can directly adapt to target domains rather than being specialized for a specific domain or scenario. Further, we investigate a transformer baseline called Swin-TESTR to focus on solving scene-text spotting for both regular and arbitrary-shaped scene text along with an exhaustive evaluation. The results clearly demonstrate the potential of intermediate representations to achieve significant performance on text spotting benchmarks across multiple domains (e.g. language, synth-to-real, and documents). both in terms of accuracy and efficiency.