Advancing Hate Speech Detection with Transformers: Insights from the MetaHate

TL;DR

This paper explores transformer-based models for hate speech detection on a large, diverse social media dataset, demonstrating that ELECTRA outperforms other models with high accuracy and analyzing key challenges like sarcasm and coded language.

Contribution

It provides a comprehensive evaluation of multiple transformer models on the large MetaHate dataset, highlighting ELECTRA's superior performance and analyzing common classification errors.

Findings

ELECTRA achieved the highest F1 score of 0.8980.

Transformer models outperform traditional RNNs, LSTMs, and CNNs.

Challenges include detecting sarcasm, coded language, and handling label noise.

Abstract

Hate speech is a widespread and harmful form of online discourse, encompassing slurs and defamatory posts that can have serious social, psychological, and sometimes physical impacts on targeted individuals and communities. As social media platforms such as X (formerly Twitter), Facebook, Instagram, Reddit, and others continue to facilitate widespread communication, they also become breeding grounds for hate speech, which has increasingly been linked to real-world hate crimes. Addressing this issue requires the development of robust automated methods to detect hate speech in diverse social media environments. Deep learning approaches, such as vanilla recurrent neural networks (RNNs), long short-term memory (LSTM), and convolutional neural networks (CNNs), have achieved good results, but are often limited by issues such as long-term dependencies and inefficient parallelization. This study represents the comprehensive exploration of transformer-based models for hate speech detection using the MetaHate dataset--a meta-collection of 36 datasets with 1.2 million social media samples. We evaluate multiple state-of-the-art transformer models, including BERT, RoBERTa, GPT-2, and ELECTRA, with fine-tuned ELECTRA achieving the highest performance (F1 score: 0.8980). We also analyze classification errors, revealing challenges with sarcasm, coded language, and label noise.