# A Large-Scale Comparison of Historical Text Normalization Systems

## TL;DR

This paper conducts the largest comparative study of historical text normalization techniques across eight languages, evaluating various methods and analyzing the impact of training data and evaluation approaches.

## Contribution

It provides a comprehensive survey and empirical comparison of rule-based, statistical, and neural normalization methods on multiple datasets, clarifying their relative strengths.

## Key findings

- Neural models perform competitively with traditional methods.
- Training data quantity significantly affects normalization accuracy.
- Evaluation methods influence the perceived performance of systems.

## Abstract

There is no consensus on the state-of-the-art approach to historical text normalization. Many techniques have been proposed, including rule-based methods, distance metrics, character-based statistical machine translation, and neural encoder--decoder models, but studies have used different datasets, different evaluation methods, and have come to different conclusions. This paper presents the largest study of historical text normalization done so far. We critically survey the existing literature and report experiments on eight languages, comparing systems spanning all categories of proposed normalization techniques, analysing the effect of training data quantity, and using different evaluation methods. The datasets and scripts are made publicly available.