Emojis as Anchors to Detect Arabic Offensive Language and Hate Speech

TL;DR

This paper presents a language-independent emoji-based method for collecting offensive and hate speech tweets, applies it to Arabic, and benchmarks transformer models, revealing cultural differences and model limitations in understanding nuanced offensive content.

Contribution

The paper introduces a novel emoji-based data collection method for offensive language detection, creates the largest Arabic dataset, and evaluates transformer models across multiple datasets.

Findings

Emojis serve as reliable anchors for offensive content across cultures.

Transformer models achieve competitive but imperfect results, often missing cultural nuances.

The dataset reveals common offensive words, targets, and patterns in hate speech.

Abstract

We introduce a generic, language-independent method to collect a large percentage of offensive and hate tweets regardless of their topics or genres. We harness the extralinguistic information embedded in the emojis to collect a large number of offensive tweets. We apply the proposed method on Arabic tweets and compare it with English tweets - analysing key cultural differences. We observed a constant usage of these emojis to represent offensiveness throughout different timespans on Twitter. We manually annotate and publicly release the largest Arabic dataset for offensive, fine-grained hate speech, vulgar and violence content. Furthermore, we benchmark the dataset for detecting offensiveness and hate speech using different transformer architectures and perform in-depth linguistic analysis. We evaluate our models on external datasets - a Twitter dataset collected using a completely different method, and a multi-platform dataset containing comments from Twitter, YouTube and Facebook, for assessing generalization capability. Competitive results on these datasets suggest that the data collected using our method captures universal characteristics of offensive language. Our findings also highlight the common words used in offensive communications, common targets for hate speech, specific patterns in violence tweets; and pinpoint common classification errors that can be attributed to limitations of NLP models. We observe that even state-of-the-art transformer models may fail to take into account culture, background and context or understand nuances present in real-world data such as sarcasm.