Ground Truth Generation for Multilingual Historical NLP using LLMs

TL;DR

This paper demonstrates how large language models can generate ground-truth annotations for historical texts in French and Chinese, enabling the fine-tuning of NLP tools to improve performance on under-resourced, domain-specific corpora.

Contribution

It introduces a method for using LLM-generated annotations to enhance NLP models for historical and low-resource languages, addressing data scarcity issues.

Findings

LLM-generated annotations improve POS, lemmatization, and NER accuracy.

Fine-tuning spaCy with synthetic data yields significant performance gains.

Domain-specific models outperform generic models on historical texts.

Abstract

Historical and low-resource NLP remains challenging due to limited annotated data and domain mismatches with modern, web-sourced corpora. This paper outlines our work in using large language models (LLMs) to create ground-truth annotations for historical French (16th-20th centuries) and Chinese (1900-1950) texts. By leveraging LLM-generated ground truth on a subset of our corpus, we were able to fine-tune spaCy to achieve significant gains on period-specific tests for part-of-speech (POS) annotations, lemmatization, and named entity recognition (NER). Our results underscore the importance of domain-specific models and demonstrate that even relatively limited amounts of synthetic data can improve NLP tools for under-resourced corpora in computational humanities research.