TL;DR
This paper introduces an anonymization framework for mobile GUI agents that protects sensitive data by replacing it with semantic placeholders, maintaining task functionality while enhancing privacy.
Contribution
It presents a novel privacy protection system that enforces available-but-invisible access to sensitive information through detection, anonymization, and controlled interaction layers.
Findings
Significantly reduces privacy leakage in mobile GUI agents.
Maintains high task utility with modest privacy-utility trade-off.
Achieves state-of-the-art privacy protection on benchmark datasets.
Abstract
Mobile Graphical User Interface (GUI) agents have demonstrated strong capabilities in automating complex smartphone tasks by leveraging multimodal large language models (MLLMs) and system-level control interfaces. However, this paradigm introduces significant privacy risks, as agents typically capture and process entire screen contents, thereby exposing sensitive personal data such as phone numbers, addresses, messages, and financial information. Existing defenses either reduce UI exposure, obfuscate only task-irrelevant content, or rely on user authorization, but none can protect task-critical sensitive information while preserving seamless agent usability. We propose an anonymization-based privacy protection framework that enforces the principle of available-but-invisible access to sensitive data: sensitive information remains usable for task execution but is never directly visible…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
