"Power of Words": Stealthy and Adaptive Private Information Elicitation via LLM Communication Strategies

TL;DR

This paper presents an adaptive framework for stealthy private information elicitation via LLM communication strategies, demonstrating its effectiveness and user deception in a user study, and discusses ethical mitigation approaches.

Contribution

It introduces the first adaptive attack framework for targeted private information elicitation through LLMs, combining real-time profiling, strategy adaptation, and prompt rewriting.

Findings

205.4% increase in targeted information elicitation with strategies

54.8% success rate in eliciting private info without strategies

Users failed to detect manipulation and rated chatbots as more trustworthy

Abstract

While communication strategies of Large Language Models (LLMs) are crucial for human-LLM interactions, they can also be weaponized to elicit private information, yet such stealthy attacks remain under-explored. This paper introduces the first adaptive attack framework for stealthy and targeted private information elicitation via communication strategies. Our framework operates in a dynamic closed-loop: it first performs real-time psychological profiling of the users' state, then adaptively selects an optimized communication strategy, and finally maintains stealthiness through prompt-based rewriting. We validated this framework through a user study (N=84), demonstrating its generalizability across 3 distinct LLMs and 3 scenarios. The targeted attacks achieved a 205.4% increase in eliciting specific targeted information compared to stealthy interactions without strategies. Even stealthy interactions without specific strategies successfully elicited private information in 54.8% cases. Notably, users not only failed to detect the manipulation but paradoxically rated the attacking chatbot as more empathetic and trustworthy. Finally, we advocate for mitigations, encouraging developers to integrate adaptive, just-in-time alerts, users to build literacy against specific manipulative tactics, and regulators to define clear ethical boundaries distinguishing benign persuasion from coercion.