Multiple Sources are Better Than One: Incorporating External Knowledge in Low-Resource Glossing

TL;DR

This paper improves automatic glossing for low-resource languages by integrating multiple external knowledge sources, including translations and large language models, resulting in significant accuracy gains especially in extremely low-resource scenarios.

Contribution

It introduces a multi-source approach combining translations and LLMs to enhance glossing accuracy in low-resource languages, outperforming previous methods.

Findings

Average 5%-point accuracy improvement across six languages.

10%-point improvement for the lowest-resourced language Gitksan.

10%-point gain in ultra-low resource settings with fewer than 100 sentences.

Abstract

In this paper, we address the data scarcity problem in automatic data-driven glossing for low-resource languages by coordinating multiple sources of linguistic expertise. We supplement models with translations at both the token and sentence level as well as leverage the extensive linguistic capability of modern LLMs. Our enhancements lead to an average absolute improvement of 5%-points in word-level accuracy over the previous state of the art on a typologically diverse dataset spanning six low-resource languages. The improvements are particularly noticeable for the lowest-resourced language Gitksan, where we achieve a 10%-point improvement. Furthermore, in a simulated ultra-low resource setting for the same six languages, training on fewer than 100 glossed sentences, we establish an average 10%-point improvement in word-level accuracy over the previous state-of-the-art system.