From Inquisitorial to Adversarial: Using Legal Theory to Redesign Online Reporting Systems

TL;DR

This paper explores redesigning online user reporting systems by applying adversarial legal models to enhance user control, privacy, and fairness, while addressing potential system abuse through innovative design strategies.

Contribution

It introduces a novel framework for integrating adversarial legal principles into online reporting systems, emphasizing user empowerment and privacy protection.

Findings

Identified the limitations of inquisitorial models in online reporting.

Proposed design strategies to empower users in evidence collection.

Discussed cryptographic tools and legal frameworks for system improvement.

Abstract

User reporting systems are central to addressing interpersonal conflicts and protecting users from harm in online spaces, particularly those with heightened privacy expectations. However, users often express frustration at their lack of insight and input into the reporting process. Drawing on offline legal literature, we trace these frustrations to the inquisitorial nature of today's online reporting systems, where moderators lead evidence gathering and case development. In contrast, adversarial models can grant users greater control and thus are better for procedural justice and privacy protection, despite their increased risks of system abuse. This motivates us to explore the potential of incorporating adversarial practices into online reporting systems. Through literature review, formative interviews, and threat modeling, we find a rich design space for empowering users to collect and present their evidence while mitigating potential abuse in the reporting process. In particular, we propose designs that minimize the amount of information shared for reporting purposes, as well as supporting evidence authentication. Finally, we discuss how our findings can inform new cryptographic tools and new efforts to apply comparative legal frameworks to online moderation.