Human-Centred LLM Privacy Audits: Findings and Frictions

TL;DR

This paper introduces LMP2, a browser-based tool for auditing LLMs' associations with personal data, revealing privacy risks, user desires for control, and challenges in evaluating probabilistic, context-dependent outputs.

Contribution

The paper presents LMP2, a novel self-audit tool for human-centered privacy assessment of LLMs, along with empirical findings and identified frictions in current evaluation methods.

Findings

GPT-4o predicts 11 of 50 features with ≥60% accuracy

Participants desire control over LLM associations

Evaluation of models shows separation between stable and default associations

Abstract

Large language models (LLMs) learn statistical associations from massive training corpora and user interactions, and deployed systems can surface or infer information about individuals. Yet people lack practical ways to inspect what a model associates with their name. We report interim findings from an ongoing study and introduce LMP2, a browser-based self-audit tool. In two user studies ($N_{total}{=}458$), GPT-4o predicts 11 of 50 features for everyday people with $\ge$60\% accuracy, and participants report wanting control over LLM-generated associations despite not considering all outputs privacy violations. To validate our probing method, we evaluate eight LLMs on public figures and non-existent names, observing clear separation between stable name-conditioned associations and model defaults. Our findings also contribute to exposing a broader generative AI evaluation crisis: when outputs are probabilistic, context-dependent, and user-mediated through elicitation, what model--individual associations even include is under-specified and operationalisation relies on crafting probes and metrics that are hard to validate or compare. To move towards reliable, actionable human-centred LLM privacy audits, we identify nine frictions that emerged in our study and offer recommendations for future work and the design of human-centred LLM privacy audits.