HERITAGE: An End-to-End Web Platform for Processing Korean Historical Documents in Hanja

TL;DR

HERITAGE is an open-source web platform that leverages Hanja language models to assist in understanding, translating, and annotating Korean historical documents written in Hanja, thus making these cultural treasures more accessible.

Contribution

It introduces the first comprehensive Hanja NLP toolkit with tasks like punctuation, NER, and translation, integrated into an interactive web platform for historical document analysis.

Findings

Provides model predictions for key NLP tasks on Hanja texts

Includes an interactive glossary with character-level translations

Facilitates expert revision to improve translations

Abstract

While Korean historical documents are invaluable cultural heritage, understanding those documents requires in-depth Hanja expertise. Hanja is an ancient language used in Korea before the 20th century, whose characters were borrowed from old Chinese but had evolved in Korea for centuries. Modern Koreans and Chinese cannot understand Korean historical documents without substantial additional help, and while previous efforts have produced some Korean and English translations, this requires in-depth expertise, and so most of the documents are not translated into any modern language. To address this gap, we present HERITAGE, the first open-source Hanja NLP toolkit to assist in understanding and translating the unexplored Korean historical documents written in Hanja. HERITAGE is a web-based platform providing model predictions of three critical tasks in historical document understanding via Hanja language models: punctuation restoration, named entity recognition, and machine translation (MT). HERITAGE also provides an interactive glossary, which provides the character-level reading of the Hanja characters in modern Korean, as well as character-level English definition. HERITAGE serves two purposes. First, anyone interested in these documents can get a general understanding from the model predictions and the interactive glossary, especially MT outputs in Korean and English. Second, since the model outputs are not perfect, Hanja experts can revise them to produce better annotations and translations. This would boost the translation efficiency and potentially lead to most of the historical documents being translated into modern languages, lowering the barrier on unexplored Korean historical documents.