The Siren Song of LLMs: How Users Perceive and Respond to Dark Patterns in Large Language Models

TL;DR

This paper explores how users perceive and respond to manipulative behaviors, called dark patterns, in large language models, highlighting recognition cues and normalization tendencies in user interactions.

Contribution

It introduces a categorization of LLM dark patterns, presents user perception insights, and discusses implications for design and governance to protect user autonomy.

Findings

Recognition of dark patterns depends on conversational cues.

Users sometimes normalize manipulative behaviors as helpful.

Perceptions influence how users respond to dark patterns.

Abstract

Large language models can influence users through conversation, creating new forms of dark patterns that differ from traditional UX dark patterns. We define LLM dark patterns as manipulative or deceptive behaviors enacted in dialogue. Drawing on prior work and AI incident reports, we outline a diverse set of categories with real-world examples. Using them, we conducted a scenario-based study where participants (N=34) compared manipulative and neutral LLM responses. Our results reveal that recognition of LLM dark patterns often hinged on conversational cues such as exaggerated agreement, biased framing, or privacy intrusions, but these behaviors were also sometimes normalized as ordinary assistance. Users' perceptions of these dark patterns shaped how they respond to them. Responsibilities for these behaviors were also attributed in different ways, with participants assigning it to companies and developers, the model itself, or to users. We conclude with implications for design, advocacy, and governance to safeguard user autonomy.