Enhancing Indic Handwritten Text Recognition Using Global Semantic Information

TL;DR

This paper introduces a novel approach for Indic handwritten text recognition that incorporates global semantic information via a semantic module, significantly improving accuracy over existing methods.

Contribution

The paper proposes integrating a semantic module based on pre-trained language models into encoder-decoder frameworks for improved Indic handwritten text recognition.

Findings

Achieves state-of-the-art results on ten Indic languages.

Enhances recognition accuracy by leveraging global semantic context.

Demonstrates robustness against image blur and incomplete characters.

Abstract

Handwritten Text Recognition (HTR) is more interesting and challenging than printed text due to uneven variations in the handwriting style of the writers, content, and time. HTR becomes more challenging for the Indic languages because of (i) multiple characters combined to form conjuncts which increase the number of characters of respective languages, and (ii) near to 100 unique basic Unicode characters in each Indic script. Recently, many recognition methods based on the encoder-decoder framework have been proposed to handle such problems. They still face many challenges, such as image blur and incomplete characters due to varying writing styles and ink density. We argue that most encoder-decoder methods are based on local visual features without explicit global semantic information. In this work, we enhance the performance of Indic handwritten text recognizers using global semantic information. We use a semantic module in an encoder-decoder framework for extracting global semantic information to recognize the Indic handwritten texts. The semantic information is used in both the encoder for supervision and the decoder for initialization. The semantic information is predicted from the word embedding of a pre-trained language model. Extensive experiments demonstrate that the proposed framework achieves state-of-the-art results on handwritten texts of ten Indic languages.