Towards a Better Modqueue: Designing for Diversity Across Moderator Objectives and Workflows

TL;DR

This paper explores the diverse objectives and workflows of Reddit moderators, highlighting the need for adaptable tools, and introduces a simulation-based approach to improve modqueue design for varied moderator needs.

Contribution

It provides a comprehensive survey of moderators' objectives and workflows and proposes a simulation method to test interventions accommodating diversity.

Findings

Moderators have varied objectives beyond accuracy, such as fairness.

Different moderators value different interventions, resisting one-size-fits-all solutions.

A simulation approach can help evaluate tradeoffs and design better moderation tools.

Abstract

Reddit relies on volunteer moderators to enforce community rules, configure tools, and review flagged content. This labor is substantial, worth millions in unpaid effort, and increasingly hard to sustain as communities grow. While recent updates to Reddit's modqueue emphasize efficiency and reducing redundancy, recent research shows that moderators use the interface in varied ways, value objectives beyond throughput (such as fairness and accuracy), and often resist features that disrupt workflows. In this paper, we survey 106 active Reddit moderators to examine the objectives they bring to their modqueue work and the kinds of interventions they consider helpful. Our findings highlight wide variation in values and workflows, with no single objective beyond accuracy dominating, and different perspectives on which interventions are useful. To address this diversity, we introduce a simulation-based approach that can complement empirical findings by probing tradeoffs and testing potential interventions, and provide design recommendations based on our findings.