MASTER: Multi-Aspect Non-local Network for Scene Text Recognition

TL;DR

MASTER introduces a self-attention based scene text recognition model that addresses attention drift, improves robustness to distortions, and enhances training and inference efficiency, outperforming existing methods on various benchmarks.

Contribution

The paper proposes a novel self-attention based architecture for scene text recognition that overcomes attention drift and improves efficiency and robustness.

Findings

Outperforms existing methods on multiple benchmarks.

Effectively handles regular and irregular scene text.

Achieves high training parallelization and fast inference.

Abstract

Attention-based scene text recognizers have gained huge success, which leverages a more compact intermediate representation to learn 1d- or 2d- attention by a RNN-based encoder-decoder architecture. However, such methods suffer from attention-drift problem because high similarity among encoded features leads to attention confusion under the RNN-based local attention mechanism. Moreover, RNN-based methods have low efficiency due to poor parallelization. To overcome these problems, we propose the MASTER, a self-attention based scene text recognizer that (1) not only encodes the input-output attention but also learns self-attention which encodes feature-feature and target-target relationships inside the encoder and decoder and (2) learns a more powerful and robust intermediate representation to spatial distortion, and (3) owns a great training efficiency because of high training parallelization and a high-speed inference because of an efficient memory-cache mechanism. Extensive experiments on various benchmarks demonstrate the superior performance of our MASTER on both regular and irregular scene text. Pytorch code can be found at https://github.com/wenwenyu/MASTER-pytorch, and Tensorflow code can be found at https://github.com/jiangxiluning/MASTER-TF.