Hate Speech and Counter Speech Detection: Conversational Context Does Matter

TL;DR

This paper demonstrates that incorporating conversational context significantly improves the detection of hate and counter speech in online comments, highlighting the importance of context-aware models for better annotation and classification.

Contribution

It introduces a new context-aware dataset for classifying hate, counter, and neutral speech on Reddit and shows that context improves neural network performance in detection tasks.

Findings

Context influences human judgment in hate speech annotation.

Neural networks perform better with context-aware inputs.

Analysis reveals when and why context helps or fails.

Abstract

Hate speech is plaguing the cyberspace along with user-generated content. This paper investigates the role of conversational context in the annotation and detection of online hate and counter speech, where context is defined as the preceding comment in a conversation thread. We created a context-aware dataset for a 3-way classification task on Reddit comments: hate speech, counter speech, or neutral. Our analyses indicate that context is critical to identify hate and counter speech: human judgments change for most comments depending on whether we show annotators the context. A linguistic analysis draws insights into the language people use to express hate and counter speech. Experimental results show that neural networks obtain significantly better results if context is taken into account. We also present qualitative error analyses shedding light into (a) when and why context is beneficial and (b) the remaining errors made by our best model when context is taken into account.