CLIOPATRA: Extracting Private Information from LLM Insights
Meenatchi Sundaram Muthu Selva Annamalai, Emiliano De Cristofaro, Peter Kairouz

TL;DR
This paper demonstrates that current heuristic privacy protections in LLM insight systems like Clio are vulnerable to sophisticated attacks, which can successfully extract sensitive user information despite layered defenses.
Contribution
It introduces CLIOPATRA, the first privacy attack against privacy-preserving LLM insight systems, revealing significant vulnerabilities in existing heuristic privacy protections.
Findings
CLIOPATRA successfully extracts medical history in 39% of cases with minimal adversary knowledge.
The attack reaches nearly 100% success with enhanced adversary knowledge and advanced models.
Existing privacy mitigations like LLM-based auditing are unreliable and often fail to detect leaks.
Abstract
As AI assistants become widely used, privacy-aware platforms like Anthropic's Clio have been introduced to generate insights from real-world AI use. Clio's privacy protections rely on layering multiple heuristic techniques together, including PII redaction, clustering, filtering, and LLM-based privacy auditing. In this paper, we put these claims to the test by presenting CLIOPATRA, the first privacy attack against "privacy-preserving" LLM insight systems. The attack involves a realistic adversary that carefully designs and inserts malicious chats into the system to break multiple layers of privacy protections and induce the leakage of sensitive information from a target user's chat. We evaluated CLIOPATRA on synthetically generated medical target chats, demonstrating that an adversary who knows only the basic demographics of a target user and a single symptom can successfully extract…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsArtificial Intelligence in Healthcare and Education · Privacy-Preserving Technologies in Data · Ethics and Social Impacts of AI
