GatedLexiconNet: A Comprehensive End-to-End Handwritten Paragraph Text Recognition System

TL;DR

GatedLexiconNet is an end-to-end handwritten paragraph recognition system that uses gated convolutional layers and attention-based line segmentation to improve accuracy over existing methods.

Contribution

It introduces gated convolutional layers and an attention mechanism for internal line segmentation in an end-to-end recognition framework, extending the existing LexiconNet model.

Findings

Achieved character error rates of 2.27% on IAM, 0.9% on RIMES, and 2.13% on READ-16 datasets.

Achieved word error rates of 5.73% on IAM, 2.76% on RIMES, and 6.52% on READ-16 datasets.

Demonstrated improved recognition performance over previous models.

Abstract

The Handwritten Text Recognition problem has been a challenge for researchers for the last few decades, especially in the domain of computer vision, a subdomain of pattern recognition. Variability of texts amongst writers, cursiveness, and different font styles of handwritten texts with degradation of historical text images make it a challenging problem. Recognizing scanned document images in neural network-based systems typically involves a two-step approach: segmentation and recognition. However, this method has several drawbacks. These shortcomings encompass challenges in identifying text regions, analyzing layout diversity within pages, and establishing accurate ground truth segmentation. Consequently, these processes are prone to errors, leading to bottlenecks in achieving high recognition accuracies. Thus, in this study, we present an end-to-end paragraph recognition system that incorporates internal line segmentation and gated convolutional layers based encoder. The gating is a mechanism that controls the flow of information and allows to adaptively selection of the more relevant features in handwritten text recognition models. The attention module plays an important role in performing internal line segmentation, allowing the page to be processed line-by-line. During the decoding step, we have integrated a connectionist temporal classification-based word beam search decoder as a post-processing step. In this work, we have extended existing LexiconNet by carefully applying and utilizing gated convolutional layers in the existing deep neural network. Our results at line and page levels also favour our new GatedLexiconNet. This study reported character error rates of 2.27% on IAM, 0.9% on RIMES, and 2.13% on READ-16, and word error rates of 5.73% on IAM, 2.76% on RIMES, and 6.52% on READ-2016 datasets.