Towards Aligning Personalized Conversational Recommendation Agents with Users' Privacy Preferences

TL;DR

This paper advocates for AI agents, especially conversational recommendation systems, to proactively align with users' privacy preferences using a new framework based on privacy theories and optimization techniques.

Contribution

It introduces a conceptual framework for aligning AI agents' privacy behaviors with user preferences, shifting from passive control to proactive alignment.

Findings

Proposes a preference alignment problem formulation.

Uses Contextual Integrity and Privacy Calculus theories.

Introduces optimization methods for privacy-utility balance.

Abstract

The proliferation of AI agents, with their complex and context-dependent actions, renders conventional privacy paradigms obsolete. This position paper argues that the current model of privacy management, rooted in a user's unilateral control over a passive tool, is inherently mismatched with the dynamic and interactive nature of AI agents. We contend that ensuring effective privacy protection necessitates that the agents proactively align with users' privacy preferences instead of passively waiting for the user to control. To ground this shift, and using personalized conversational recommendation agents as a case, we propose a conceptual framework built on Contextual Integrity (CI) theory and Privacy Calculus theory. This synthesis first reframes automatically controlling users' privacy as an alignment problem, where AI agents initially did not know users' preferences, and would learn their privacy preferences through implicit or explicit feedback. Upon receiving the preference feedback, the agents used alignment and Pareto optimization for aligning preferences and balancing privacy and utility. We introduced formulations and instantiations, potential applications, as well as five challenges.