CWoMP: Morpheme Representation Learning for Interlinear Glossing

TL;DR

CWoMP introduces a contrastive pretraining approach that models morphemes as atomic units for interlinear glossing, improving interpretability and efficiency especially in low-resource language settings.

Contribution

The paper presents CWoMP, a novel method that learns morpheme representations and aligns them with words, enhancing automated interlinear glossing with interpretability and adaptability.

Findings

Outperforms existing methods in low-resource languages

More efficient than previous approaches

Strong gains in extremely low-resource settings

Abstract

Interlinear glossed text (IGT) is a standard notation for language documentation which is linguistically rich but laborious to produce manually. Recent automated IGT methods treat glosses as character sequences, neglecting their compositional structure. We propose CWoMP (Contrastive Word-Morpheme Pretraining), which instead treats morphemes as atomic form-meaning units with learned representations. A contrastively trained encoder aligns words-in-context with their constituent morphemes in a shared embedding space; an autoregressive decoder then generates the morpheme sequence by retrieving entries from a mutable lexicon of these embeddings. Predictions are interpretable--grounded in lexicon entries--and users can improve results at inference time by expanding the lexicon without retraining. We evaluate on diverse low-resource languages, showing that CWoMP outperforms existing methods while being significantly more efficient, with particularly strong gains in extremely low-resource settings.