Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech

TL;DR

This paper evaluates ChatGPT's ability to generate natural language explanations for implicit hate speech detection, comparing its performance with human explanations to assess its potential and limitations.

Contribution

It introduces a prompt design for ChatGPT to produce explanations and conducts user studies to compare these with human-generated explanations for implicit hate speech.

Findings

ChatGPT can generate explanations that are comparable to human explanations.

There are notable limitations in ChatGPT's explanations for implicit hate speech.

The study highlights the potential of ChatGPT but also its current constraints in this domain.

Abstract

Recent studies have alarmed that many online hate speeches are implicit. With its subtle nature, the explainability of the detection of such hateful speech has been a challenging problem. In this work, we examine whether ChatGPT can be used for providing natural language explanations (NLEs) for implicit hateful speech detection. We design our prompt to elicit concise ChatGPT-generated NLEs and conduct user studies to evaluate their qualities by comparison with human-written NLEs. We discuss the potential and limitations of ChatGPT in the context of implicit hateful speech research.