A Little Annotation does a Lot of Good: A Study in Bootstrapping Low-resource Named Entity Recognizers

TL;DR

This study explores combining cross-lingual transfer and targeted annotation to efficiently develop high-quality named entity recognizers in low-resource languages, reducing annotation effort while maintaining accuracy.

Contribution

It demonstrates that a dual-strategy approach using transfer learning followed by targeted annotation outperforms traditional methods in low-resource NER tasks.

Findings

Cross-lingual transfer is highly effective with minimal annotated data.

Targeted annotation of uncertain spans achieves competitive accuracy quickly.

Combining transfer and targeted annotation reduces annotation effort by tenfold.

Abstract

Most state-of-the-art models for named entity recognition (NER) rely on the availability of large amounts of labeled data, making them challenging to extend to new, lower-resourced languages. However, there are now several proposed approaches involving either cross-lingual transfer learning, which learns from other highly resourced languages, or active learning, which efficiently selects effective training data based on model predictions. This paper poses the question: given this recent progress, and limited human annotation, what is the most effective method for efficiently creating high-quality entity recognizers in under-resourced languages? Based on extensive experimentation using both simulated and real human annotation, we find a dual-strategy approach best, starting with a cross-lingual transferred model, then performing targeted annotation of only uncertain entity spans in the target language, minimizing annotator effort. Results demonstrate that cross-lingual transfer is a powerful tool when very little data can be annotated, but an entity-targeted annotation strategy can achieve competitive accuracy quickly, with just one-tenth of training data.