Towards Robust Handwritten Text Recognition with On-the-fly User Participation

TL;DR

This paper presents a user-participation based strategy for iteratively improving handwritten Hindi OCR models through selective user data collection and curriculum learning, enhancing long-term OCR service quality.

Contribution

It introduces a novel iterative model upgrade approach incorporating user data selection and curriculum learning for robust handwritten text recognition.

Findings

Model improves over three iterations with user data selection.

Curriculum learning enhances model adaptation to diverse handwriting styles.

Selective user data contributes to better OCR performance on unseen styles.

Abstract

Long-term OCR services aim to provide high-quality output to their users at competitive costs. It is essential to upgrade the models because of the complex data loaded by the users. The service providers encourage the users who provide data where the OCR model fails by rewarding them based on data complexity, readability, and available budget. Hitherto, the OCR works include preparing the models on standard datasets without considering the end-users. We propose a strategy of consistently upgrading an existing Handwritten Hindi OCR model three times on the dataset of 15 users. We fix the budget of 4 users for each iteration. For the first iteration, the model directly trains on the dataset from the first four users. For the rest iteration, all remaining users write a page each, which service providers later analyze to select the 4 (new) best users based on the quality of predictions on the human-readable words. Selected users write 23 more pages for upgrading the model. We upgrade the model with Curriculum Learning (CL) on the data available in the current iteration and compare the subset from previous iterations. The upgraded model is tested on a held-out set of one page each from all 23 users. We provide insights into our investigations on the effect of CL, user selection, and especially the data from unseen writing styles. Our work can be used for long-term OCR services in crowd-sourcing scenarios for the service providers and end users.