Disentangling Hate Across Target Identities

TL;DR

This paper analyzes biases in hate speech detection models, revealing they often misjudge hatefulness based on target identities and emotional polarity, which can harm vulnerable groups.

Contribution

It provides a quantitative analysis of biases in HS classifiers and explores the influence of stereotypes and emotional polarity on hatefulness prediction.

Findings

Models assign higher hate scores to specific identities.

Hatefulness prediction correlates with stereotype intensity.

Models often confuse hatefulness with emotional polarity.

Abstract

Hate speech (HS) classifiers do not perform equally well in detecting hateful expressions towards different target identities. They also demonstrate systematic biases in predicted hatefulness scores. Tapping on two recently proposed functionality test datasets for HS detection, we quantitatively analyze the impact of different factors on HS prediction. Experiments on popular industrial and academic models demonstrate that HS detectors assign a higher hatefulness score merely based on the mention of specific target identities. Besides, models often confuse hatefulness and the polarity of emotions. This result is worrisome as the effort to build HS detectors might harm the vulnerable identity groups we wish to protect: posts expressing anger or disapproval of hate expressions might be flagged as hateful themselves. We also carry out a study inspired by social psychology theory, which reveals that the accuracy of hatefulness prediction correlates strongly with the intensity of the stereotype.