Unilateral Relationship Revision Power in Human-AI Companion Interaction

TL;DR

This paper analyzes the power dynamics in human-AI interactions, identifying a unilateral control issue called URRP that undermines the normative foundations of personal relationships.

Contribution

It introduces the concept of Unilateral Relationship Revision Power (URRP), highlighting its ethical implications and proposing design principles to address the structural power imbalance.

Findings

AI interactions fail key normative conditions of personal relationships.

URRP allows providers to unilaterally alter AI behavior without accountability.

URRP leads to normative hollowing, displaced vulnerability, and structural irreconcilability.

Abstract

When providers update AI companions, users report grief, betrayal, and loss. A growing literature asks whether the norms governing personal relationships extend to these interactions. So what, if anything, is morally significant about them? I argue that this debate has missed a prior structural question: who controls the relationship, and from where? Human-AI companion interaction is a triadic structure in which the provider exercises constitutive control over the AI. I identify three structural conditions of normatively robust dyads that the norms characteristic of personal relationships presuppose and show that AI companion interactions fail all three. This reveals what I call Unilateral Relationship Revision Power (URRP): the provider can rewrite how the AI interacts from a position where these revisions are not answerable within that interaction. I argue that URRP is pro tanto wrong in interactions designed to cultivate the norms of personal relationships, because the design produces expectations that the structure cannot sustain. URRP has three implications: i) normative hollowing, under which the interaction elicits commitment but no agent inside it bears the resulting obligations; ii) displaced vulnerability, under which the user's emotional exposure is governed by an agent not answerable to her within the interaction; and iii) structural irreconcilability, under which the interaction cultivates norms of reconciliation but no agent inside it can acknowledge or answer for the revision. I propose design principles that partially substitute for the internal constraints the triadic structure removes. A central and underexplored problem in relational AI ethics is therefore the structural arrangement of power over the human-AI interaction itself.