HateBench: Benchmarking Hate Speech Detectors on LLM-Generated Content and Hate Campaigns

TL;DR

HateBench evaluates hate speech detectors against LLM-generated hate speech, revealing performance degradation with newer LLMs and exposing new threats from LLM-driven hate campaigns using adversarial techniques.

Contribution

This paper introduces HateBench, a comprehensive benchmark for hate speech detection on LLM-generated content, and uncovers vulnerabilities and new threats posed by advanced attack methods.

Findings

Detectors perform well but decline with newer LLMs

LLMs can be exploited to generate and evade hate speech detection

Adversarial and model stealing attacks significantly increase attack success rates

Abstract

Large Language Models (LLMs) have raised increasing concerns about their misuse in generating hate speech. Among all the efforts to address this issue, hate speech detectors play a crucial role. However, the effectiveness of different detectors against LLM-generated hate speech remains largely unknown. In this paper, we propose HateBench, a framework for benchmarking hate speech detectors on LLM-generated hate speech. We first construct a hate speech dataset of 7,838 samples generated by six widely-used LLMs covering 34 identity groups, with meticulous annotations by three labelers. We then assess the effectiveness of eight representative hate speech detectors on the LLM-generated dataset. Our results show that while detectors are generally effective in identifying LLM-generated hate speech, their performance degrades with newer versions of LLMs. We also reveal the potential of LLM-driven hate campaigns, a new threat that LLMs bring to the field of hate speech detection. By leveraging advanced techniques like adversarial attacks and model stealing attacks, the adversary can intentionally evade the detector and automate hate campaigns online. The most potent adversarial attack achieves an attack success rate of 0.966, and its attack efficiency can be further improved by $13-21\times$ through model stealing attacks with acceptable attack performance. We hope our study can serve as a call to action for the research community and platform moderators to fortify defenses against these emerging threats.