Offensive Hebrew Corpus and Detection using BERT

TL;DR

This paper introduces a new Hebrew offensive language dataset from Twitter, and demonstrates how fine-tuning BERT models on this data improves offensive language detection accuracy, addressing low-resource language challenges.

Contribution

The paper presents a novel Hebrew offensive language corpus and evaluates BERT-based models, enhancing detection performance and offering insights into low-resource language NLP.

Findings

Our dataset improves HeBERT performance by 2%.

Fine-tuning AlephBERT on our data achieves 69% accuracy.

The dataset shows potential for generalizability across datasets.

Abstract

Offensive language detection has been well studied in many languages, but it is lagging behind in low-resource languages, such as Hebrew. In this paper, we present a new offensive language corpus in Hebrew. A total of 15,881 tweets were retrieved from Twitter. Each was labeled with one or more of five classes (abusive, hate, violence, pornographic, or none offensive) by Arabic-Hebrew bilingual speakers. The annotation process was challenging as each annotator is expected to be familiar with the Israeli culture, politics, and practices to understand the context of each tweet. We fine-tuned two Hebrew BERT models, HeBERT and AlephBERT, using our proposed dataset and another published dataset. We observed that our data boosts HeBERT performance by 2% when combined with D_OLaH. Fine-tuning AlephBERT on our data and testing on D_OLaH yields 69% accuracy, while fine-tuning on D_OLaH and testing on our data yields 57% accuracy, which may be an indication to the generalizability our data offers. Our dataset and fine-tuned models are available on GitHub and Huggingface.