A hypothesize-and-verify framework for Text Recognition using Deep Recurrent Neural Networks

TL;DR

This paper introduces a hypothesize-and-verify framework for text recognition that combines deep recurrent neural networks with a language model to improve accuracy by verifying multiple segmentation hypotheses.

Contribution

It presents a novel hybrid text recognition system that integrates multi-hypotheses segmentation with deep RNNs and language model verification to enhance recognition accuracy.

Findings

Improved recognition accuracy through hypothesis verification.

Effective elimination of segmentation errors.

Robust handling of long-range context in text recognition.

Abstract

Deep LSTM is an ideal candidate for text recognition. However text recognition involves some initial image processing steps like segmentation of lines and words which can induce error to the recognition system. Without segmentation, learning very long range context is difficult and becomes computationally intractable. Therefore, alternative soft decisions are needed at the pre-processing level. This paper proposes a hybrid text recognizer using a deep recurrent neural network with multiple layers of abstraction and long range context along with a language model to verify the performance of the deep neural network. In this paper we construct a multi-hypotheses tree architecture with candidate segments of line sequences from different segmentation algorithms at its different branches. The deep neural network is trained on perfectly segmented data and tests each of the candidate segments, generating unicode sequences. In the verification step, these unicode sequences are validated using a sub-string match with the language model and best first search is used to find the best possible combination of alternative hypothesis from the tree structure. Thus the verification framework using language models eliminates wrong segmentation outputs and filters recognition errors.