Open Source Handwritten Text Recognition on Medieval Manuscripts using Mixed Models and Document-Specific Finetuning

TL;DR

This paper presents open source mixed models for handwritten text recognition on medieval German manuscripts, achieving low error rates out-of-the-box and effective finetuning for improved accuracy with minimal data.

Contribution

The authors develop and release mixed recognition models for medieval manuscripts that perform well without training and can be finetuned efficiently for specific scripts.

Findings

Out-of-the-box CER of 6.22% on unseen manuscripts.

Finetuning reduces CER to below 2%.

Finetuning out-of-domain models outperforms training from scratch.

Abstract

This paper deals with the task of practical and open source Handwritten Text Recognition (HTR) on German medieval manuscripts. We report on our efforts to construct mixed recognition models which can be applied out-of-the-box without any further document-specific training but also serve as a starting point for finetuning by training a new model on a few pages of transcribed text (ground truth). To train the mixed models we collected a corpus of 35 manuscripts and ca. 12.5k text lines for two widely used handwriting styles, Gothic and Bastarda cursives. Evaluating the mixed models out-of-the-box on four unseen manuscripts resulted in an average Character Error Rate (CER) of 6.22%. After training on 2, 4 and eventually 32 pages the CER dropped to 3.27%, 2.58%, and 1.65%, respectively. While the in-domain recognition and training of models (Bastarda model to Bastarda material, Gothic to Gothic) unsurprisingly yielded the best results, finetuning out-of-domain models to unseen scripts was still shown to be superior to training from scratch. Our new mixed models have been made openly available to the community.