Multilinguals at SemEval-2022 Task 11: Complex NER in Semantically Ambiguous Settings for Low Resource Languages

TL;DR

This paper explores the use of pre-trained language models with Whole Word Masking for complex Named Entity Recognition in low-resource Chinese and Spanish, achieving significant improvements over baselines.

Contribution

It introduces the application of Whole Word Masking with various neural architectures to enhance NER performance in low-resource languages.

Findings

All models outperform baseline significantly

Best model achieves competitive leaderboard position

Whole Word Masking boosts masked language modeling in low-resource settings

Abstract

We leverage pre-trained language models to solve the task of complex NER for two low-resource languages: Chinese and Spanish. We use the technique of Whole Word Masking(WWM) to boost the performance of masked language modeling objective on large and unsupervised corpora. We experiment with multiple neural network architectures, incorporating CRF, BiLSTMs, and Linear Classifiers on top of a fine-tuned BERT layer. All our models outperform the baseline by a significant margin and our best performing model obtains a competitive position on the evaluation leaderboard for the blind test set.