Handwritten Text Recognition for Low Resource Languages

TL;DR

This paper presents BharatOCR, a novel segmentation-free paragraph-level handwritten text recognition system for low-resource languages like Hindi and Urdu, utilizing a ViT-Transformer architecture with language models to improve accuracy and coherence.

Contribution

It introduces a new ViT-Transformer Decoder-LM architecture specifically designed for paragraph-level handwritten recognition in low-resource languages, with a focus on implicit line segmentation.

Findings

Achieved character recognition rates of 96.24% in Urdu and 94.80% in Hindi datasets.

Outperformed several state-of-the-art Urdu text recognition methods.

Provided benchmark results on multiple datasets, demonstrating high accuracy.

Abstract

Despite considerable progress in handwritten text recognition, paragraph-level handwritten text recognition, especially in low-resource languages, such as Hindi, Urdu and similar scripts, remains a challenging problem. These languages, often lacking comprehensive linguistic resources, require special attention to develop robust systems for accurate optical character recognition (OCR). This paper introduces BharatOCR, a novel segmentation-free paragraph-level handwritten Hindi and Urdu text recognition. We propose a ViT-Transformer Decoder-LM architecture for handwritten text recognition, where a Vision Transformer (ViT) extracts visual features, a Transformer decoder generates text sequences, and a pre-trained language model (LM) refines the output to improve accuracy, fluency, and coherence. Our model utilizes a Data-efficient Image Transformer (DeiT) model proposed for masked image modeling in this research work. In addition, we adopt a RoBERTa architecture optimized for masked language modeling (MLM) to enhance the linguistic comprehension and generative capabilities of the proposed model. The transformer decoder generates text sequences from visual embeddings. This model is designed to iteratively process a paragraph image line by line, called implicit line segmentation. The proposed model was evaluated using our custom dataset ('Parimal Urdu') and ('Parimal Hindi'), introduced in this research work, as well as two public datasets. The proposed model achieved benchmark results in the NUST-UHWR, PUCIT-OUHL, and Parimal-Urdu datasets, achieving character recognition rates of 96.24%, 92.05%, and 94.80%, respectively. The model also provided benchmark results using the Hindi dataset achieving a character recognition rate of 80.64%. The results obtained from our proposed model indicated that it outperformed several state-of-the-art Urdu text recognition methods.