Lexicon and Attention based Handwritten Text Recognition System

TL;DR

This paper introduces a novel handwritten text recognition system that combines lexicon and attention mechanisms with neural networks, achieving significant accuracy improvements on standard datasets.

Contribution

The study merges attention mechanisms with existing neural network architectures for handwritten text recognition, demonstrating notable error rate reductions.

Findings

Achieved 4.15% character error rate on IAM dataset

Achieved 7.07% character error rate on GW dataset

Improved character error rate by 23.27% using attention with greedy decoding

Abstract

The handwritten text recognition problem is widely studied by the researchers of computer vision community due to its scope of improvement and applicability to daily lives, It is a sub-domain of pattern recognition. Due to advancement of computational power of computers since last few decades neural networks based systems heavily contributed towards providing the state-of-the-art handwritten text recognizers. In the same direction, we have taken two state-of-the art neural networks systems and merged the attention mechanism with it. The attention technique has been widely used in the domain of neural machine translations and automatic speech recognition and now is being implemented in text recognition domain. In this study, we are able to achieve 4.15% character error rate and 9.72% word error rate on IAM dataset, 7.07% character error rate and 16.14% word error rate on GW dataset after merging the attention and word beam search decoder with existing Flor et al. architecture. To analyse further, we have also used system similar to Shi et al. neural network system with greedy decoder and observed 23.27% improvement in character error rate from the base model.