Profiling for Pennies: Unveiling the Privacy Iceberg of LLM Agents
Jiahao Chen, Qi Zhang, Ruixiao Lin, Chunyi Zhou, Tianyu Du, Qingming Li, Tong Zhang, Junhao Li, Yuwen Pu, Shouling Ji

TL;DR
This paper investigates privacy risks associated with LLMs, revealing a dissonance between public concerns and platform practices, and introduces the PrivacyIceberg framework and IcebergExplorer tool for systematic privacy risk assessment.
Contribution
It introduces the PrivacyIceberg framework to categorize real-world privacy risks and develops IcebergExplorer, a tool to audit privacy exposure with high accuracy and low cost.
Findings
Platforms fail to address public privacy concerns effectively.
IcebergExplorer achieves over 90% factual accuracy in reconstructing profiles.
Identifies six root causes of privacy disclosures.
Abstract
Large Language Models (LLMs) have revolutionized how information are collected, aggregated, and reasoned. However, this enables a novel and accessible vector of privacy intrusion: the automated and in-depth personal profiling; this engenders a chilling effect of "peepers everywhere". Existing research primarily unfolds from the training pipeline of LLM, emphasizing the exposure of Personally Identifiable Information (PII) through memorization, while privacy studies from a human-centric perspective remain underexplored. To fill this void, we empirically investigate privacy perception in the real world through the lens of human awareness and the practices of LLM-integrated platforms, revealing a significant dissonance: platforms fail to technically or policy-wise address public privacy concerns. To facilitate a systematic and quantifiable study of privacy risk, we propose the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
