Scene Text Recognition with Sliding Convolutional Character Models

TL;DR

This paper introduces a scene text recognition method using sliding convolutional character models that detects and recognizes characters without segmentation, trained end-to-end, and capable of recognizing unknown words efficiently.

Contribution

The proposed method combines sliding window detection with CNN-based character models and CTC decoding, avoiding RNNs and segmentation, enabling fast, accurate, and lexicon-free scene text recognition.

Findings

Achieves superior or comparable performance on multiple benchmarks.

Avoids character segmentation and RNN training issues.

Enables fast, parallel recognition of unknown words.

Abstract

Scene text recognition has attracted great interests from the computer vision and pattern recognition community in recent years. State-of-the-art methods use concolutional neural networks (CNNs), recurrent neural networks with long short-term memory (RNN-LSTM) or the combination of them. In this paper, we investigate the intrinsic characteristics of text recognition, and inspired by human cognition mechanisms in reading texts, we propose a scene text recognition method with character models on convolutional feature map. The method simultaneously detects and recognizes characters by sliding the text line image with character models, which are learned end-to-end on text line images labeled with text transcripts. The character classifier outputs on the sliding windows are normalized and decoded with Connectionist Temporal Classification (CTC) based algorithm. Compared to previous methods, our method has a number of appealing properties: (1) It avoids the difficulty of character segmentation which hinders the performance of segmentation-based recognition methods; (2) The model can be trained simply and efficiently because it avoids gradient vanishing/exploding in training RNN-LSTM based models; (3) It bases on character models trained free of lexicon, and can recognize unknown words. (4) The recognition process is highly parallel and enables fast recognition. Our experiments on several challenging English and Chinese benchmarks, including the IIIT-5K, SVT, ICDAR03/13 and TRW15 datasets, demonstrate that the proposed method yields superior or comparable performance to state-of-the-art methods while the model size is relatively small.