Cleaning Up the Streets: Understanding Motivations, Mental Models, and Concerns of Users Flagging Social Media Content

TL;DR

This study investigates social media users' motivations, mental models, and concerns regarding content flagging, revealing transparency gaps and proposing design improvements to better support users and address online harm.

Contribution

It provides new insights into user perceptions and mental models of flagging mechanisms, emphasizing transparency and privacy issues, and suggests design innovations for social media platforms.

Findings

Users lack procedural transparency in flagging mechanisms.

Users strongly believe in the importance of flagging options.

Flagging raises questions about responsibility and labor distribution.

Abstract

Social media platforms offer flagging, a technical feature that empowers users to report inappropriate posts or bad actors to reduce online harm. The deceptively simple flagging interfaces on nearly all major social media platforms disguise complex underlying interactions among users, algorithms, and moderators. Through interviewing 25 social media users with prior flagging experience, most of whom belong to marginalized groups, we examine end-users' understanding of flagging procedures, explore the factors that motivate them to flag, and surface their cognitive and privacy concerns. We found that a lack of procedural transparency in flagging mechanisms creates gaps in users' mental models, yet they strongly believe that platforms must provide flagging options. Our findings highlight how flags raise critical questions about distributing labor and responsibility between platforms and users for addressing online harm. We recommend innovations in the flagging design space that enhance user comprehension, ensure privacy, and reduce cognitive burdens.