Measuring and Forecasting Conversation Incivility: the Role of Antisocial and Prosocial Behaviors

TL;DR

This paper develops new metrics to measure conversation incivility after hate speech replies, analyzes language patterns, and forecasts incivility levels, aiming to promote civil discourse and prevent escalation.

Contribution

It introduces novel metrics for incivility measurement, improves alignment with human perception, and analyzes language and interaction patterns related to antisocial and prosocial behaviors.

Findings

New metrics better align with human perceptions

Language patterns differ between antisocial and prosocial posts

Forecasting incivility remains a challenging task

Abstract

This paper focuses on the task of measuring and forecasting incivility in conversations following replies to hate speech. Identifying replies that steer conversations away from hatred and elicit civil follow-up conversations sheds light into effective strategies to engage with hate speech and proactively avoid further escalation. We propose new metrics that take into account various dimensions of antisocial and prosocial behaviors to measure the conversation incivility following replies to hate speech. Our best metric aligns with human perceptions better than prior work. Additionally, we present analyses on a) the language of antisocial and prosocial posts, b) the relationship between antisocial or prosocial posts and user interactions, and c) the language of replies to hate speech that elicit follow-up conversations with different incivility levels. We show that forecasting the incivility level of conversations following a reply to hate speech is a challenging task. We also present qualitative analyses to identify the most common errors made by our best model.