Detection of Hate Speech using BERT and Hate Speech Word Embedding with Deep Model

TL;DR

This paper explores the use of domain-specific word embeddings and BERT in deep learning models to improve the automatic detection and classification of online hate speech, achieving high accuracy.

Contribution

It introduces the application of domain-specific word embeddings with BiLSTM and leverages BERT for hate speech detection, demonstrating their effectiveness.

Findings

Domain-specific embeddings with BiLSTM achieved 93% F1-score.

BERT achieved up to 96% F1-score.

Models effectively detect hate speech despite obfuscation tactics.

Abstract

The enormous amount of data being generated on the web and social media has increased the demand for detecting online hate speech. Detecting hate speech will reduce their negative impact and influence on others. A lot of effort in the Natural Language Processing (NLP) domain aimed to detect hate speech in general or detect specific hate speech such as religion, race, gender, or sexual orientation. Hate communities tend to use abbreviations, intentional spelling mistakes, and coded words in their communication to evade detection, adding more challenges to hate speech detection tasks. Thus, word representation will play an increasingly pivotal role in detecting hate speech. This paper investigates the feasibility of leveraging domain-specific word embedding in Bidirectional LSTM based deep model to automatically detect/classify hate speech. Furthermore, we investigate the use of the transfer learning language model (BERT) on hate speech problem as a binary classification task. The experiments showed that domainspecific word embedding with the Bidirectional LSTM based deep model achieved a 93% f1-score while BERT achieved up to 96% f1-score on a combined balanced dataset from available hate speech datasets.