From Traditional Taggers to LLMs: A Comparative Study of POS Tagging for Medieval Romance Languages

TL;DR

This study evaluates large language models for POS tagging in medieval Romance languages, showing they outperform traditional methods, especially with transfer learning, and provides practical guidance for digital humanities applications.

Contribution

It offers a systematic empirical comparison of LLMs and traditional taggers for medieval languages, highlighting transfer learning strategies and releasing resources for reproducibility.

Findings

LLMs outperform traditional POS taggers across medieval languages.

Fine-tuning and multilingual training significantly improve LLM performance.

Cross-lingual transfer learning benefits under-resourced medieval language varieties.

Abstract

Part-of-speech (POS) tagging for Medieval Romance languages remains challenging due to orthographic variation, morphological complexity, and limited annotated resources. This paper presents a systematic empirical evaluation of large language models (LLMs) for POS tagging across three medieval varieties: Medieval Occitan, Medieval Catalan, and Medieval French. We compare traditional rule-based and statistical taggers with modern open-source LLMs under zero-shot prompting, few-shot prompting, monolingual fine-tuning, and cross-lingual transfer learning settings. Experiments on historically grounded datasets show that LLM-based approaches consistently outperform traditional taggers, with fine-tuning and multilingual training yielding the largest improvements. In particular, cross-lingual transfer learning substantially benefits under-resourced varieties, while targeted bilingual training can outperform broader multilingual configurations for specific target languages. The results highlight the importance of linguistic proximity and dataset characteristics when designing transfer strategies for historical NLP. These findings provide empirical insights into the applicability of modern neural methods to medieval text processing and provide practical guidance for deploying LLM-based POS tagging pipelines in digital humanities research. All code, models, and processed datasets are released for reproducibility.