"Did I Say Something Wrong?" A Word-Level Analysis of Wikipedia Articles for Deletion Discussions

TL;DR

This study analyzes Wikipedia deletion discussions at the word level to identify linguistic markers of constructive versus disruptive messages, revealing that 'You'-messages strongly indicate disruption, while 'I'-messages also unexpectedly correlate with disruption.

Contribution

It introduces a large-scale, automated dataset of annotated discussion messages and applies classifiers to identify linguistic indicators of discussion quality, highlighting the roles of pronouns and function words.

Findings

'You'-messages are strong indicators of disruptive communication.

'I'-messages also correlate with disruptive messages, contrary to expectations.

Function words' importance remains inconclusive.

Abstract

This thesis focuses on gaining linguistic insights into textual discussions on a word level. It was of special interest to distinguish messages that constructively contribute to a discussion from those that are detrimental to them. Thereby, we wanted to determine whether "I"- and "You"-messages are indicators for either of the two discussion styles. These messages are nowadays often used in guidelines for successful communication. Although their effects have been successfully evaluated multiple times, a large-scale analysis has never been conducted. Thus, we used Wikipedia Articles for Deletion (short: AfD) discussions together with the records of blocked users and developed a fully automated creation of an annotated data set. In this data set, messages were labelled either constructive or disruptive. We applied binary classifiers to the data to determine characteristic words for both discussion styles. Thereby, we also investigated whether function words like pronouns and conjunctions play an important role in distinguishing the two. We found that "You"-messages were a strong indicator for disruptive messages which matches their attributed effects on communication. However, we found "I"-messages to be indicative for disruptive messages as well which is contrary to their attributed effects. The importance of function words could neither be confirmed nor refuted. Other characteristic words for either communication style were not found. Yet, the results suggest that a different model might represent disruptive and constructive messages in textual discussions better.