I Beg to Differ: A study of constructive disagreement in online conversations

TL;DR

This study analyzes constructive disagreement in online conversations, introducing a new dataset and models to predict escalation to moderation, highlighting the importance of linguistic dynamics and conversation structure.

Contribution

The paper introduces WikiDisputes, a new corpus of Wikipedia disputes, and develops neural models that leverage conversation structure and linguistic changes to predict escalation.

Findings

Neural models outperform feature-based models in prediction accuracy.

Incorporating conversation structure improves model performance.

Model accuracy increases and uncertainty decreases with more conversation data.

Abstract

Disagreements are pervasive in human communication. In this paper we investigate what makes disagreement constructive. To this end, we construct WikiDisputes, a corpus of 7 425 Wikipedia Talk page conversations that contain content disputes, and define the task of predicting whether disagreements will be escalated to mediation by a moderator. We evaluate feature-based models with linguistic markers from previous work, and demonstrate that their performance is improved by using features that capture changes in linguistic markers throughout the conversations, as opposed to averaged values. We develop a variety of neural models and show that taking into account the structure of the conversation improves predictive accuracy, exceeding that of feature-based models. We assess our best neural model in terms of both predictive accuracy and uncertainty by evaluating its behaviour when it is only exposed to the beginning of the conversation, finding that model accuracy improves and uncertainty reduces as models are exposed to more information.