Handling Heavily Abbreviated Manuscripts: HTR engines vs text normalisation approaches

TL;DR

This paper investigates methods for normalizing heavily abbreviated medieval Latin manuscripts, comparing direct handwritten text recognition (HTR) approaches with multi-step processes involving specialized models for recognition and normalization.

Contribution

It introduces and evaluates different computational setups for expanding abbreviations in medieval Latin manuscripts, highlighting the effectiveness of combined recognition and normalization models.

Findings

Direct HTR on normalized text improves accuracy.

Multi-step approaches with specialist models enhance abbreviation expansion.

Case studies demonstrate practical effectiveness in medieval Latin texts.

Abstract

Although abbreviations are fairly common in handwritten sources, particularly in medieval and modern Western manuscripts, previous research dealing with computational approaches to their expansion is scarce. Yet abbreviations present particular challenges to computational approaches such as handwritten text recognition and natural language processing tasks. Often, pre-processing ultimately aims to lead from a digitised image of the source to a normalised text, which includes expansion of the abbreviations. We explore different setups to obtain such a normalised text, either directly, by training HTR engines on normalised (i.e., expanded, disabbreviated) text, or by decomposing the process into discrete steps, each making use of specialist models for recognition, word segmentation and normalisation. The case studies considered here are drawn from the medieval Latin tradition.