Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs

TL;DR

This paper introduces TRACE-RPS, a novel framework combining fine-grained anonymization and inference-preventing optimization to significantly reduce attribute inference accuracy in LLMs, enhancing user privacy.

Contribution

The paper presents a unified defense framework that improves privacy protection against attribute inference attacks in LLMs by integrating attention-based anonymization with inference rejection strategies.

Findings

Reduces attribute inference accuracy from ~50% to below 5% on open-source models.

Demonstrates strong cross-model generalization and robustness to prompt variations.

Maintains utility-privacy tradeoffs effectively.

Abstract

Recent studies have shown that large language models (LLMs) can infer private user attributes (e.g., age, location, gender) from user-generated text shared online, enabling rapid and large-scale privacy breaches. Existing anonymization-based defenses are coarse-grained, lacking word-level precision in anonymizing privacy-leaking elements. Moreover, they are inherently limited as altering user text to hide sensitive cues still allows attribute inference to occur through models' reasoning capabilities. To address these limitations, we propose a unified defense framework that combines fine-grained anonymization (TRACE) with inference-preventing optimization (RPS). TRACE leverages attention mechanisms and inference chain generation to identify and anonymize privacy-leaking textual elements, while RPS employs a lightweight two-stage optimization strategy to induce model rejection behaviors, thereby preventing attribute inference. Evaluations across diverse LLMs show that TRACE-RPS reduces attribute inference accuracy from around 50\% to below 5\% on open-source models. In addition, our approach offers strong cross-model generalization, prompt-variation robustness, and utility-privacy tradeoffs. Our code is available at https://github.com/Jasper-Yan/TRACE-RPS.