Breaking BERT: Understanding its Vulnerabilities for Named Entity Recognition through Adversarial Attack

TL;DR

This paper investigates the vulnerabilities of BERT-based models for Named Entity Recognition by applying adversarial attacks, revealing significant sensitivity to local context changes and emergent entities, especially in domain-specific models.

Contribution

It provides a comprehensive analysis of BERT models' vulnerabilities to adversarial input variations in NER tasks, highlighting the need for robustness improvements.

Findings

BERT models are highly vulnerable to local context changes in NER.

Single modifications can often fool the models.

Domain-specific models like SciBERT are more vulnerable than general BERT.

Abstract

Both generic and domain-specific BERT models are widely used for natural language processing (NLP) tasks. In this paper we investigate the vulnerability of BERT models to variation in input data for Named Entity Recognition (NER) through adversarial attack. Experimental results show that BERT models are vulnerable to variation in the entity context with 20.2 to 45.0% of entities predicted completely wrong and another 29.3 to 53.3% of entities predicted wrong partially. BERT models seem most vulnerable to changes in the local context of entities and often a single change is sufficient to fool the model. The domain-specific BERT model trained from scratch (SciBERT) is more vulnerable than the original BERT model or the domain-specific model that retains the BERT vocabulary (BioBERT). We also find that BERT models are particularly vulnerable to emergent entities. Our results chart the vulnerabilities of BERT models for NER and emphasize the importance of further research into uncovering and reducing these weaknesses.