Adversarial Adaptation for French Named Entity Recognition

TL;DR

This paper introduces an adversarial adaptation method for French NER using Transformer models, leveraging unlabeled corpora to improve generalization and performance in resource-limited settings.

Contribution

It presents a novel adversarial adaptation framework for Transformer-based French NER that effectively utilizes unlabeled data to enhance model robustness and accuracy.

Findings

Outperforms non-adaptive models across multiple datasets

Reduces overfitting by using large-scale unlabeled corpora

Mitigates performance dip of smaller pre-trained Transformer models

Abstract

Named Entity Recognition (NER) is the task of identifying and classifying named entities in large-scale texts into predefined classes. NER in French and other relatively limited-resource languages cannot always benefit from approaches proposed for languages like English due to a dearth of large, robust datasets. In this paper, we present our work that aims to mitigate the effects of this dearth of large, labeled datasets. We propose a Transformer-based NER approach for French, using adversarial adaptation to similar domain or general corpora to improve feature extraction and enable better generalization. Our approach allows learning better features using large-scale unlabeled corpora from the same domain or mixed domains to introduce more variations during training and reduce overfitting. Experimental results on three labeled datasets show that our adaptation framework outperforms the corresponding non-adaptive models for various combinations of Transformer models, source datasets, and target corpora. We also show that adversarial adaptation to large-scale unlabeled corpora can help mitigate the performance dip incurred on using Transformer models pre-trained on smaller corpora.