Building Low-Resource NER Models Using Non-Speaker Annotation

TL;DR

This paper introduces a novel approach for low-resource NER by utilizing non-speaker annotations, demonstrating that non-native annotators can produce results comparable or superior to cross-lingual methods, especially with further effort.

Contribution

It presents a new method using non-speaker annotations for low-resource NER, showing its effectiveness compared to existing cross-lingual approaches.

Findings

Non-speaker annotations match or outperform cross-lingual methods.

Participants successfully annotated Indonesian, Russian, and Hindi.

Additional effort can further improve non-speaker annotation results.

Abstract

In low-resource natural language processing (NLP), the key problems are a lack of target language training data, and a lack of native speakers to create it. Cross-lingual methods have had notable success in addressing these concerns, but in certain common circumstances, such as insufficient pre-training corpora or languages far from the source language, their performance suffers. In this work we propose a complementary approach to building low-resource Named Entity Recognition (NER) models using ``non-speaker'' (NS) annotations, provided by annotators with no prior experience in the target language. We recruit 30 participants in a carefully controlled annotation experiment with Indonesian, Russian, and Hindi. We show that use of NS annotators produces results that are consistently on par or better than cross-lingual methods built on modern contextual representations, and have the potential to outperform with additional effort. We conclude with observations of common annotation patterns and recommended implementation practices, and motivate how NS annotations can be used in addition to prior methods for improved performance. For more details, http://cogcomp.org/page/publication_view/941