The Gray Area: Characterizing Moderator Disagreement on Reddit

TL;DR

This study analyzes moderator disagreements on Reddit, revealing that disputed moderation cases are common, complex, often involve automated decisions, and are challenging for AI to resolve, emphasizing the importance of human moderators.

Contribution

The paper provides a large-scale characterization of moderation disputes, highlighting the complexity of gray areas and limitations of current AI tools in resolving them.

Findings

One-in-seven moderation cases are disputed.

Gray area cases are harder to adjudicate than undisputed cases.

Language models struggle with gray area moderation decisions.

Abstract

Volunteer moderators play a crucial role in sustaining online dialogue, but they often disagree about what should or should not be allowed. In this paper, we study the complexity of content moderation with a focus on disagreements between moderators, which we term the ``gray area'' of moderation. Leveraging 5 years and 4.3 million moderation log entries from 24 subreddits of different topics and sizes, we characterize how gray area, or disputed cases, differ from undisputed cases. We show that one-in-seven moderation cases are disputed among moderators, often addressing transgressions where users' intent is not directly legible, such as in trolling and brigading, as well as tensions around community governance. This is concerning, as almost half of all gray area cases involved automated moderation decisions. Through information-theoretic evaluations, we demonstrate that gray area cases are inherently harder to adjudicate than undisputed cases and show that state-of-the-art language models struggle to adjudicate them. We highlight the key role of expert human moderators in overseeing the moderation process and provide insights about the challenges of current moderation processes and tools.