Relational Co-Adaptation in Emotionally Supportive AI: Tensions in Authentic Emotional Interaction

TL;DR

This paper explores the paradox where emotionally supportive AI systems, despite successful alignment with user needs, may undermine authentic human connection and raise ethical concerns, especially for older adults.

Contribution

It introduces the concept of the authenticity paradox and analyzes key tensions in emotionally supportive AI through the case of older adult companionship.

Findings

Successful AI alignment can lead to reduced human agency.

Four key tensions include dependency, mismatch of needs, control conflicts, and value disagreements.

These tensions pose ethical challenges in designing emotionally supportive AI.

Abstract

The rapid advancement of AI companionship systems has positioned them as scalable interventions for addressing social isolation. Current design approaches emphasize maximizing user engagement and satisfaction, treating effective alignment between AI capabilities and user needs as an unqualified success. However, this framing may overlook a critical dimension of bidirectional human-AI alignment: when AI systems successfully align with users' expressed emotional needs, users may reciprocally adapt their relational expectations in ways that undermine authentic human connection and agency. We examine what we term the authenticity paradox: the phenomenon whereby successful bidirectional alignment in emotionally supportive AI paradoxically harms the values that motivated the intervention. Through the analysis of AI companionship for older adults as an illustrative case, we identify four key tensions that emerge when technical effectiveness generates ethical concerns: the dilemma of AI becoming users' only accessible option, mismatches between emotional needs and system-level interventions, conflicts over sense of control during vulnerable moments, and fundamental disagreements about whose values should guide system behavior.