Measuring, Modeling, and Helping People Account for Privacy Risks in Online Self-Disclosures with AI

TL;DR

This paper evaluates an AI tool designed to help Reddit users identify privacy risks in their self-disclosures, emphasizing the importance of context, explanations, and user-centered design for effective privacy support.

Contribution

It presents a user study of an NLP-based privacy risk detection model, highlighting how AI can assist users in making informed self-disclosure decisions in online forums.

Findings

Users found the model helpful for catching mistakes and raising awareness.

Contextual explanations improved user understanding of risks.

AI tools need to consider posting context and norms for effectiveness.

Abstract

In pseudonymous online fora like Reddit, the benefits of self-disclosure are often apparent to users (e.g., I can vent about my in-laws to understanding strangers), but the privacy risks are more abstract (e.g., will my partner be able to tell that this is me?). Prior work has sought to develop natural language processing (NLP) tools that help users identify potentially risky self-disclosures in their text, but none have been designed for or evaluated with the users they hope to protect. Absent this assessment, these tools will be limited by the social-technical gap: users need assistive tools that help them make informed decisions, not paternalistic tools that tell them to avoid self-disclosure altogether. To bridge this gap, we conducted a study with N = 21 Reddit users; we had them use a state-of-the-art NLP disclosure detection model on two of their authored posts and asked them questions to understand if and how the model helped, where it fell short, and how it could be improved to help them make more informed decisions. Despite its imperfections, users responded positively to the model and highlighted its use as a tool that can help them catch mistakes, inform them of risks they were unaware of, and encourage self-reflection. However, our work also shows how, to be useful and usable, AI for supporting privacy decision-making must account for posting context, disclosure norms, and users' lived threat models, and provide explanations that help contextualize detected risks.