Adaptive Embedding Gate for Attention-Based Scene Text Recognition

TL;DR

This paper introduces the adaptive embedding gate (AEG), a novel module for attention-based scene text recognition that improves accuracy and robustness by better modeling character dependencies.

Contribution

The paper proposes the adaptive embedding gate (AEG), a new module that enhances attention mechanisms with high-order language modeling for scene text recognition.

Findings

AEG significantly improves recognition accuracy on multiple benchmarks.

AEG enhances robustness against challenging scene text conditions.

AEG can be integrated into existing attention-based models with ease.

Abstract

Scene text recognition has attracted particular research interest because it is a very challenging problem and has various applications. The most cutting-edge methods are attentional encoder-decoder frameworks that learn the alignment between the input image and output sequences. In particular, the decoder recurrently outputs predictions, using the prediction of the previous step as a guidance for every time step. In this study, we point out that the inappropriate use of previous predictions in existing attention mechanisms restricts the recognition performance and brings instability. To handle this problem, we propose a novel module, namely adaptive embedding gate(AEG). The proposed AEG focuses on introducing high-order character language models to attention mechanism by controlling the information transmission between adjacent characters. AEG is a flexible module and can be easily integrated into the state-of-the-art attentional methods. We evaluate its effectiveness as well as robustness on a number of standard benchmarks, including the IIIT$5$K, SVT, SVT-P, CUTE$80$, and ICDAR datasets. Experimental results demonstrate that AEG can significantly boost recognition performance and bring better robustness.