Venire: A Machine Learning-Guided Panel Review System for Community Content Moderation

TL;DR

Venire is an ML-guided panel review system designed to improve consistency and resolve disagreements in community content moderation by identifying contentious cases for multi-person review, enhancing moderation quality.

Contribution

The paper introduces Venire, a novel ML-backed system that predicts moderator disagreements to facilitate multi-person review, addressing decision inconsistency in community moderation.

Findings

Venire improves decision consistency in moderation.

Venire helps surface and resolve latent disagreements.

Moderators gain confidence in difficult cases using Venire.

Abstract

Research into community content moderation often assumes that moderation teams govern with a single, unified voice. However, recent work has found that moderators disagree with one another at modest, but concerning rates. The problem is not the root disagreements themselves. Subjectivity in moderation is unavoidable, and there are clear benefits to including diverse perspectives within a moderation team. Instead, the crux of the issue is that, due to resource constraints, moderation decisions end up being made by individual decision-makers. The result is decision-making that is inconsistent, which is frustrating for community members. To address this, we develop Venire, an ML-backed system for panel review on Reddit. Venire uses a machine learning model trained on log data to identify the cases where moderators are most likely to disagree. Venire fast-tracks these cases for multi-person review. Ideally, Venire allows moderators to surface and resolve disagreements that would have otherwise gone unnoticed. We conduct three studies through which we design and evaluate Venire: a set of formative interviews with moderators, technical evaluations on two datasets, and a think-aloud study in which moderators used Venire to make decisions on real moderation cases. Quantitatively, we demonstrate that Venire is able to improve decision consistency and surface latent disagreements. Qualitatively, we find that Venire helps moderators resolve difficult moderation cases more confidently. Venire represents a novel paradigm for human-AI content moderation, and shifts the conversation from replacing human decision-making to supporting it.