Hate Cannot Drive out Hate: Forecasting Conversation Incivility following Replies to Hate Speech

TL;DR

This paper investigates whether replies to hate speech can predict and prevent incivility in subsequent conversations, proposing a new metric and analyzing linguistic features to improve understanding and forecasting of conversation civility.

Contribution

It introduces a novel metric for measuring conversation incivility and evaluates models predicting incivility following replies to hate speech, highlighting challenges and linguistic factors involved.

Findings

The proposed metric aligns well with human judgments.

Forecasting incivility remains a challenging task.

Linguistic differences in replies influence subsequent civility levels.

Abstract

User-generated replies to hate speech are promising means to combat hatred, but questions about whether they can stop incivility in follow-up conversations linger. We argue that effective replies stop incivility from emerging in follow-up conversations - replies that elicit more incivility are counterproductive. This study introduces the task of predicting the incivility of conversations following replies to hate speech. We first propose a metric to measure conversation incivility based on the number of civil and uncivil comments as well as the unique authors involved in the discourse. Our metric approximates human judgments more accurately than previous metrics. We then use the metric to evaluate the outcomes of replies to hate speech. A linguistic analysis uncovers the differences in the language of replies that elicit follow-up conversations with high and low incivility. Experimental results show that forecasting incivility is challenging. We close with a qualitative analysis shedding light into the most common errors made by the best model.