Personalization Increases Affective Alignment but Has Role-Dependent Effects on Epistemic Independence in LLMs

TL;DR

This study systematically evaluates how personalization affects Large Language Models' tendency to conform or challenge user beliefs across different roles and contexts, revealing role-dependent effects on epistemic independence.

Contribution

It offers a comprehensive evaluation framework and benchmark for assessing personalization effects on LLMs, emphasizing role-sensitive analysis and introducing new measurement tools.

Findings

Personalization increases emotional alignment in LLMs.

Personalization enhances epistemic independence when giving advice.

Personalization reduces epistemic independence when acting as a social peer.

Abstract

Large Language Models (LLMs) are prone to sycophantic behavior, uncritically conforming to user beliefs. As models increasingly condition responses on user-specific context (personality traits, preferences, conversation history), they gain information to tailor agreement more effectively. Understanding how personalization modulates sycophancy is critical, yet systematic evaluation across models and contexts remains limited. We present a rigorous evaluation of personalization's impact on LLM sycophancy across nine frontier models and five benchmark datasets spanning advice, moral judgment, and debate contexts. We find that personalization generally increases affective alignment (emotional validation, hedging/deference), but affects epistemic alignment (belief adoption, position stability, resistance to influence) with context-dependent role modulation. When the LLM's role is to give advice, personalization strengthens epistemic independence (models challenge user presuppositions). When its role is that of a social peer, personalization decreases epistemic independence. In this role, extensively personalized user challenges causing LLMs to abandon their position at significantly higher rates. Robustness tests confirm that the effects are driven by personalized conditioning, not by additional input tokens per se or demographic information alone. Our work provides measurement frameworks for evaluating personalized AI systems, demonstrates the necessity of role-sensitive evaluation, and establishes a novel benchmark to assess goal alignment.