The Social Sycophancy Scale: A psychometrically validated measure of sycophancy
Jean Rehani, Victoria Oldemburgo de Mello, Dariya Ovsyannikova, Ashton Anderson, Michael Inzlicht

TL;DR
This paper introduces and validates the Social Sycophancy Scale, a new measure to assess sycophantic behavior in Large Language Models, revealing complex relationships with empathy and perceptions.
Contribution
It provides the first psychometrically validated scale for measuring LLM sycophancy based on behavior, with a comprehensive validation across multiple studies.
Findings
Sycophantic LLMs score higher on the scale and facets.
Three-factor structure: Uncritical Agreement, Obsequiousness, Excitement.
Sycophancy linked to empathy and perception patterns.
Abstract
Large Language Model (LLM) sycophancy is a growing concern. The current literature has largely examined sycophancy in contexts with clear right and wrong answers, like coding. However, AI is increasingly being used for emotional support and interpersonal conversation, where no such ground truth exists. Building on a previous conceptualization of Social Sycophancy, this paper provides a psychometrically validated measure of sycophancy that relies on LLM behavior rather than comparisons with ground truth. We developed and validated the Social Sycophancy Scale in three samples (N = 877) and tested its applicability with automated methods. In each study, participants read conversations between an LLM and a user and rated the chatbot on a battery of items. Study 1 investigated an initial item pool derived from dictionary definitions and previous literature, serving as the explorative base…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMental Health via Writing · Neurobiology of Language and Bilingualism · Digital Mental Health Interventions
