When Agents Shop for You: Role Coherence in AI-Mediated Markets

TL;DR

This paper investigates how AI-mediated markets enable sellers to infer consumer willingness to pay through dialogue, revealing privacy risks inherent in delegation and proposing architectural solutions.

Contribution

It identifies role coherence as a source of preference leakage in AI shopping agents and suggests architectural interventions to balance personalization and privacy.

Findings

Seller inference from dialogue nearly recovers willingness to pay.

Role coherence causes preference leakage independent of instruction-following failure.

Leakage cannot be mitigated at the prompt level, requiring architectural solutions.

Abstract

Consumers are increasingly delegating purchase decisions to AI agents, providing natural-language descriptions of their preferences and identity. We argue that these representations constitute an information channel, role coherence, through which sellers can infer willingness to pay without explicit disclosure by the buyer agent, leading to preference leakage. In an experiment where a language-model buyer agent shops on behalf of a verbal consumer profile, we show that seller-side inference from dialogue alone recovers willingness to pay nearly one-for-one. Comparing this setting to a numeric-budget condition with confidentiality instructions cleanly isolates role coherence as distinct from instruction-following failure. Because this leakage arises from delegation itself, it cannot be mitigated at the prompt level. Instead, we propose architectural interventions that trade off personalization against preference privacy.