Unsupervised Adaptation for Synthetic-to-Real Handwritten Word Recognition

TL;DR

This paper introduces an unsupervised writer adaptation method that adjusts a synthetic-trained handwritten word recognizer to new writers, effectively handling diverse styles and document conditions without manual annotation.

Contribution

The proposed approach enables automatic adaptation of a generic recognizer to new writers using unsupervised learning, improving robustness across varied datasets and challenging conditions.

Findings

Maintains performance across diverse datasets

Handles modern and historic documents effectively

No manual annotation required for adaptation

Abstract

Handwritten Text Recognition (HTR) is still a challenging problem because it must deal with two important difficulties: the variability among writing styles, and the scarcity of labelled data. To alleviate such problems, synthetic data generation and data augmentation are typically used to train HTR systems. However, training with such data produces encouraging but still inaccurate transcriptions in real words. In this paper, we propose an unsupervised writer adaptation approach that is able to automatically adjust a generic handwritten word recognizer, fully trained with synthetic fonts, towards a new incoming writer. We have experimentally validated our proposal using five different datasets, covering several challenges (i) the document source: modern and historic samples, which may involve paper degradation problems; (ii) different handwriting styles: single and multiple writer collections; and (iii) language, which involves different character combinations. Across these challenging collections, we show that our system is able to maintain its performance, thus, it provides a practical and generic approach to deal with new document collections without requiring any expensive and tedious manual annotation step.