Improving Self-training for Cross-lingual Named Entity Recognition with Contrastive and Prototype Learning

TL;DR

This paper introduces ContProto, a novel framework combining contrastive self-training and prototype-based pseudo-labeling to enhance cross-lingual NER performance by improving representation quality and label accuracy.

Contribution

It proposes a unified approach that improves pseudo-label quality and cross-lingual transferability in self-training for NER.

Findings

Significant performance gains over state-of-the-art methods

Effective pseudo-label refinement improves NER accuracy

Enhanced cross-lingual transferability of representations

Abstract

In cross-lingual named entity recognition (NER), self-training is commonly used to bridge the linguistic gap by training on pseudo-labeled target-language data. However, due to sub-optimal performance on target languages, the pseudo labels are often noisy and limit the overall performance. In this work, we aim to improve self-training for cross-lingual NER by combining representation learning and pseudo label refinement in one coherent framework. Our proposed method, namely ContProto mainly comprises two components: (1) contrastive self-training and (2) prototype-based pseudo-labeling. Our contrastive self-training facilitates span classification by separating clusters of different classes, and enhances cross-lingual transferability by producing closely-aligned representations between the source and target language. Meanwhile, prototype-based pseudo-labeling effectively improves the accuracy of pseudo labels during training. We evaluate ContProto on multiple transfer pairs, and experimental results show our method brings in substantial improvements over current state-of-the-art methods.