Transformer-based HTR for Historical Documents

TL;DR

This paper demonstrates that the TrOCR transformer framework effectively transcribes historical manuscripts, outperforming state-of-the-art systems like Transkribus, with minimal training data and easy adaptation to multiple Latin-based languages.

Contribution

It shows that TrOCR is a strong, adaptable model for historical handwritten text recognition, surpassing existing methods without requiring baseline information.

Findings

TrOCR outperforms Transkribus in HTR tasks.

TrOCR adapts easily to multiple Latin-based languages.

Minimal training data needed for effective transfer learning.

Abstract

We apply the TrOCR framework to real-world, historical manuscripts and show that TrOCR per se is a strong model, ideal for transfer learning. TrOCR has been trained on English only, but it can adapt to other languages that use the Latin alphabet fairly easily and with little training material. We compare TrOCR against a SOTA HTR framework (Transkribus) and show that it can beat such systems. This finding is essential since Transkribus performs best when it has access to baseline information, which is not needed at all to fine-tune TrOCR.