SilverAlign: MT-Based Silver Data Algorithm For Evaluating Word Alignment

TL;DR

SilverAlign is a novel method that automatically generates silver data for evaluating word aligners using machine translation and minimal pairs, effectively addressing the lack of gold data especially in low-resource languages.

Contribution

It introduces a new automatic approach to create evaluation data for word aligners, enabling assessment without gold standards across multiple languages.

Findings

Performance on silver data correlates well with gold benchmarks.

Effective for 9 language pairs.

Useful for low-resource language evaluation.

Abstract

Word alignments are essential for a variety of NLP tasks. Therefore, choosing the best approaches for their creation is crucial. However, the scarce availability of gold evaluation data makes the choice difficult. We propose SilverAlign, a new method to automatically create silver data for the evaluation of word aligners by exploiting machine translation and minimal pairs. We show that performance on our silver data correlates well with gold benchmarks for 9 language pairs, making our approach a valid resource for evaluation of different domains and languages when gold data are not available. This addresses the important scenario of missing gold data alignments for low-resource languages.