A Holistic Representation Guided Attention Network for Scene Text Recognition

TL;DR

This paper introduces a simple, efficient scene text recognition model that directly connects 2D CNN features to an attention decoder guided by holistic representations, eliminating the need for recurrent modules and achieving state-of-the-art results.

Contribution

The proposed model removes recurrent modules, enabling parallel training and inference, and uses holistic representations to guide attention, improving accuracy on irregular scene text recognition.

Findings

Achieves 1.5x to 9.4x faster backward pass

Achieves 1.3x to 7.9x faster forward pass

State-of-the-art or competitive performance on benchmarks

Abstract

Reading irregular scene text of arbitrary shape in natural images is still a challenging problem, despite the progress made recently. Many existing approaches incorporate sophisticated network structures to handle various shapes, use extra annotations for stronger supervision, or employ hard-to-train recurrent neural networks for sequence modeling. In this work, we propose a simple yet strong approach for scene text recognition. With no need to convert input images to sequence representations, we directly connect two-dimensional CNN features to an attention-based sequence decoder which guided by holistic representation. The holistic representation can guide the attention-based decoder focus on more accurate area. As no recurrent module is adopted, our model can be trained in parallel. It achieves 1.5x to 9.4x acceleration to backward pass and 1.3x to 7.9x acceleration to forward pass, compared with the RNN counterparts. The proposed model is trained with only word-level annotations. With this simple design, our method achieves state-of-the-art or competitive recognition performance on the evaluated regular and irregular scene text benchmark datasets.