Statistical Analysis of Perspective Scores on Hate Speech Detection

TL;DR

This paper statistically analyzes Perspective Scores in hate speech detection, revealing dataset similarities and demonstrating that oversampling these scores enhances model generalization across diverse datasets.

Contribution

It introduces a statistical analysis of Perspective Scores and shows that oversampling these scores improves hate speech detection generalization.

Findings

Different hate speech datasets have similar Perspective Scores.

Oversampling Perspective Scores significantly improves cross-dataset performance.

High-level features like Perspective Scores reduce dataset bias.

Abstract

Hate speech detection has become a hot topic in recent years due to the exponential growth of offensive language in social media. It has proven that, state-of-the-art hate speech classifiers are efficient only when tested on the data with the same feature distribution as training data. As a consequence, model architecture plays the second role to improve the current results. In such a diverse data distribution relying on low level features is the main cause of deficiency due to natural bias in data. That's why we need to use high level features to avoid a biased judgement. In this paper, we statistically analyze the Perspective Scores and their impact on hate speech detection. We show that, different hate speech datasets are very similar when it comes to extract their Perspective Scores. Eventually, we prove that, over-sampling the Perspective Scores of a hate speech dataset can significantly improve the generalization performance when it comes to be tested on other hate speech datasets.