Strategic Filtering for Content Moderation: Free Speech or Free of Distortion?

TL;DR

This paper explores how to design automated content moderation systems that balance free speech and social distortion, proposing practical approximation methods and analyzing data requirements for effective moderation.

Contribution

It introduces a mechanism design framework for optimizing content moderation trade-offs and provides approximation algorithms with generalization guarantees.

Findings

Proposes a mechanism design approach for moderation trade-offs

Develops practical approximation algorithms for optimal moderation

Provides data requirements for effective moderation approximation

Abstract

User-generated content (UGC) on social media platforms is vulnerable to incitements and manipulations, necessitating effective regulations. To address these challenges, those platforms often deploy automated content moderators tasked with evaluating the harmfulness of UGC and filtering out content that violates established guidelines. However, such moderation inevitably gives rise to strategic responses from users, who strive to express themselves within the confines of guidelines. Such phenomena call for a careful balance between: 1. ensuring freedom of speech -- by minimizing the restriction of expression; and 2. reducing social distortion -- measured by the total amount of content manipulation. We tackle the problem of optimizing this balance through the lens of mechanism design, aiming at optimizing the trade-off between minimizing social distortion and maximizing free speech. Although determining the optimal trade-off is NP-hard, we propose practical methods to approximate the optimal solution. Additionally, we provide generalization guarantees determining the amount of finite offline data required to approximate the optimal moderator effectively.