How to disagree well: Investigating the dispute tactics used on Wikipedia

TL;DR

This paper introduces a unified framework for dispute tactics in Wikipedia disagreements, annotates a large dataset, and develops models to predict tactics, improving understanding and prediction of dispute escalation.

Contribution

It proposes a comprehensive dispute tactics framework, annotates Wikipedia disagreements, and develops transformer-based models with auxiliary tasks for better tactic prediction.

Findings

Transformer-based models outperform baselines

Adding ordering information improves prediction accuracy

Annotations help predict escalation in disputes

Abstract

Disagreements are frequently studied from the perspective of either detecting toxicity or analysing argument structure. We propose a framework of dispute tactics that unifies these two perspectives, as well as other dialogue acts which play a role in resolving disputes, such as asking questions and providing clarification. This framework includes a preferential ordering among rebuttal-type tactics, ranging from ad hominem attacks to refuting the central argument. Using this framework, we annotate 213 disagreements (3,865 utterances) from Wikipedia Talk pages. This allows us to investigate research questions around the tactics used in disagreements; for instance, we provide empirical validation of the approach to disagreement recommended by Wikipedia. We develop models for multilabel prediction of dispute tactics in an utterance, achieving the best performance with a transformer-based label powerset model. Adding an auxiliary task to incorporate the ordering of rebuttal tactics further yields a statistically significant increase. Finally, we show that these annotations can be used to provide useful additional signals to improve performance on the task of predicting escalation.