Unveiling Factors for Enhanced POS Tagging: A Study of Low-Resource Medieval Romance Languages

TL;DR

This paper investigates the challenges and solutions for POS tagging in low-resource Medieval Romance languages, highlighting the impact of linguistic evolution and spelling variations on modern language models.

Contribution

It systematically evaluates various techniques like fine-tuning, prompt engineering, and transfer learning to improve POS tagging in historical languages.

Findings

LLMs face limitations with historical language variations

Specialized techniques improve tagging accuracy

Cross-lingual transfer shows promise for low-resource languages

Abstract

Part-of-speech (POS) tagging remains a foundational component in natural language processing pipelines, particularly critical for historical text analysis at the intersection of computational linguistics and digital humanities. Despite significant advancements in modern large language models (LLMs) for ancient languages, their application to Medieval Romance languages presents distinctive challenges stemming from diachronic linguistic evolution, spelling variations, and labeled data scarcity. This study systematically investigates the central determinants of POS tagging performance across diverse corpora of Medieval Occitan, Medieval Spanish, and Medieval French texts, spanning biblical, hagiographical, medical, and dietary domains. Through rigorous experimentation, we evaluate how fine-tuning approaches, prompt engineering, model architectures, decoding strategies, and cross-lingual transfer learning techniques affect tagging accuracy. Our results reveal both notable limitations in LLMs' ability to process historical language variations and non-standardized spelling, as well as promising specialized techniques that effectively address the unique challenges presented by low-resource historical languages.