Arabic Synonym BERT-based Adversarial Examples for Text Classification
Norah Alshahrani, Saied Alshahrani, Esma Wali, Jeanna Matthews

TL;DR
This paper investigates the vulnerability of Arabic text classification models, especially BERT, to synonym-based adversarial attacks, and evaluates defense strategies like adversarial training.
Contribution
It introduces the first word-level adversarial attack study for Arabic using BERT and assesses model robustness, transferability, and defense mechanisms.
Findings
Fine-tuned BERT models are more vulnerable to synonym attacks.
Transferred adversarial examples are more effective on fine-tuned BERT.
Adversarial training improves BERT model accuracy by at least 2%.
Abstract
Text classification systems have been proven vulnerable to adversarial text examples, modified versions of the original text examples that are often unnoticed by human eyes, yet can force text classification models to alter their classification. Often, research works quantifying the impact of adversarial text attacks have been applied only to models trained in English. In this paper, we introduce the first word-level study of adversarial attacks in Arabic. Specifically, we use a synonym (word-level) attack using a Masked Language Modeling (MLM) task with a BERT model in a black-box setting to assess the robustness of the state-of-the-art text classification models to adversarial attacks in Arabic. To evaluate the grammatical and semantic similarities of the newly produced adversarial examples using our synonym BERT-based attack, we invite four human evaluators to assess and compare the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
- 🤗NorahAlshahrani/BERThardmodel· 5 dl5 dl
- 🤗NorahAlshahrani/BERTmsdamodel· 9 dl9 dl
- 🤗NorahAlshahrani/biLSTMhardmodel· 13 dl13 dl
- 🤗NorahAlshahrani/biLSTMmsdamodel· 6 dl6 dl
- 🤗NorahAlshahrani/2dCNNhardmodel· 9 dl9 dl
- 🤗NorahAlshahrani/2dCNNmsdamodel· 6 dl6 dl
- 🤗NorahAlshahrani/Adv_BERT_Hardmodel· 3 dl· ♡ 13 dl♡ 1
- 🤗NorahAlshahrani/Adv_BERT_msdamodel· 1 dl1 dl
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · Attention Is All You Need · Linear Layer · Multi-Head Attention · Adam · Residual Connection · Attention Dropout · Dropout · Layer Normalization · Dense Connections
