Automated Profile Inference with Language Model Agents

TL;DR

This paper demonstrates how large language models can be exploited to automatically infer sensitive personal information from online activities, highlighting a new privacy threat and proposing an automated profiling framework.

Contribution

The paper introduces AutoProfiler, a novel framework that shows the feasibility of automated profile inference attacks using LLMs, revealing significant privacy risks.

Findings

AutoProfiler effectively infers sensitive attributes from user data.

The attack is efficient and works on real-world datasets.

Inferred attributes are both identifiable and sensitive.

Abstract

Impressive progress has been made in automated problem-solving by the collaboration of large language model (LLM) based agents. However, these automated capabilities also open avenues for malicious applications. In this paper, we study a new threat that LLMs pose to online pseudonymity, called automated profile inference, where an adversary can instruct LLMs to automatically collect and extract sensitive personal attributes from publicly available user activities on pseudonymous platforms. We also introduce an automated profiling framework called AutoProfiler to demonstrate and assess the feasibility of such attacks in real-world scenarios. AutoProfiler consists of four specialized LLM agents that work collaboratively to retrieve and process user online activities and generate a profile with extracted personal information. Experimental results on two real-world datasets and one synthetic dataset show that AutoProfiler is highly effective and efficient, and the inferred attributes are both identifiable and sensitive, posing significant privacy risks. We explore mitigation strategies from different perspectives and advocate for increased public awareness of this emerging privacy threat.