Hate Speech Dataset from a White Supremacy Forum

TL;DR

This paper introduces a new hate speech dataset from a white supremacist forum, detailing its creation, annotation process, and baseline classification experiments, to aid research in hate speech detection.

Contribution

It presents a manually labeled hate speech dataset from Stormfront, along with an annotation tool and baseline classification results, advancing resources for hate speech detection research.

Findings

The dataset contains thousands of labeled sentences.

Baseline models achieve varying levels of accuracy.

The dataset is publicly available for research use.

Abstract

Hate speech is commonly defined as any communication that disparages a target group of people based on some characteristic such as race, colour, ethnicity, gender, sexual orientation, nationality, religion, or other characteristic. Due to the massive rise of user-generated web content on social media, the amount of hate speech is also steadily increasing. Over the past years, interest in online hate speech detection and, particularly, the automation of this task has continuously grown, along with the societal impact of the phenomenon. This paper describes a hate speech dataset composed of thousands of sentences manually labelled as containing hate speech or not. The sentences have been extracted from Stormfront, a white supremacist forum. A custom annotation tool has been developed to carry out the manual labelling task which, among other things, allows the annotators to choose whether to read the context of a sentence before labelling it. The paper also provides a thoughtful qualitative and quantitative study of the resulting dataset and several baseline experiments with different classification models. The dataset is publicly available.