Transfer Learning via Lexical Relatedness: A Sarcasm and Hate Speech Case Study

TL;DR

This paper investigates whether pre-training models on sarcasm detection can enhance the performance of hate speech detection, demonstrating that sarcasm pre-training improves recall, AUC, F1-score, and precision in identifying hate speech.

Contribution

It introduces a novel transfer learning approach using sarcasm detection as a pre-training step to improve hate speech detection models, especially for implicit cases.

Findings

Sarcasm pre-training increased BERT+BiLSTM recall by 9.7%.

Pre-training improved AUC by 7.8%.

Precision on implicit hate speech increased by 7.8%.

Abstract

Detecting hate speech in non-direct forms, such as irony, sarcasm, and innuendos, remains a persistent challenge for social networks. Although sarcasm and hate speech are regarded as distinct expressions, our work explores whether integrating sarcasm as a pre-training step improves implicit hate speech detection and, by extension, explicit hate speech detection. Incorporating samples from ETHOS, Sarcasm on Reddit, and Implicit Hate Corpus, we devised two training strategies to compare the effectiveness of sarcasm pre-training on a CNN+LSTM and BERT+BiLSTM model. The first strategy is a single-step training approach, where a model trained only on sarcasm is then tested on hate speech. The second strategy uses sequential transfer learning to fine-tune models for sarcasm, implicit hate, and explicit hate. Our results show that sarcasm pre-training improved the BERT+BiLSTM's recall by 9.7%, AUC by 7.8%, and F1-score by 6% on ETHOS. On the Implicit Hate Corpus, precision increased by 7.8% when tested only on implicit samples. By incorporating sarcasm into the training process, we show that models can more effectively detect both implicit and explicit hate.