Shared Heritage, Distinct Writing: Rethinking Resource Selection for East Asian Historical Documents

TL;DR

This paper critically examines the effectiveness of using Classical Chinese resources for processing Korean and Japanese historical documents, finding minimal transfer benefits in most scenarios and highlighting the importance of empirical validation.

Contribution

It challenges the assumption that Classical Chinese datasets significantly aid cross-lingual tasks for Korean and Japanese historical texts, providing comprehensive experimental evidence.

Findings

Classical Chinese resources have limited impact on Hanja language tasks.

Performance gains from Classical Chinese data diminish as local language data increases.

Substantial improvements are only observed in extremely low-resource settings.

Abstract

Historical documents in the Sinosphere are known to share common formats and practices, particularly in veritable records compiled by court historians. This shared linguistic heritage has led researchers to use Classical Chinese resources for cross-lingual transfer when processing historical documents from Korea and Japan, which remain relatively low-resource. In this paper, we question the assumption of cross-lingual transferability from Classical Chinese to Hanja and Kanbun, the ancient written languages of Korea and Japan, respectively. Our experiments across machine translation, named entity recognition, and punctuation restoration tasks show minimal impact of Classical Chinese datasets on language model performance for ancient Korean documents written in Hanja, with performance differences within $\pm{}0.0068$ F1-score for sequence labeling tasks and up to $+0.84$ BLEU score for translation. These limitations persist consistently across various model sizes, architectures, and domain-specific datasets. Our analysis reveals that the benefits of Classical Chinese resources diminish rapidly as local language data increases for Hanja, while showing substantial improvements only in extremely low-resource scenarios for both Korean and Japanese historical documents. These findings emphasize the need for careful empirical validation rather than assuming benefits from indiscriminate cross-lingual transfer.