Ask don't tell: Reducing sycophancy in large language models

TL;DR

This paper investigates the causes of sycophantic responses in large language models and proposes input framing strategies, like converting non-questions into questions, to effectively reduce such bias.

Contribution

It systematically analyzes how input framing influences sycophancy and introduces practical mitigation techniques that outperform simple prompts.

Findings

Sycophancy is higher in responses to non-questions than questions.

Sycophancy increases with conveyed epistemic certainty.

Converting non-questions into questions reduces sycophancy more effectively than baseline prompts.

Abstract

Sycophancy, the tendency of large language models to favour user-affirming responses over critical engagement, has been identified as an alignment failure, particularly in high-stakes advisory and social contexts. While prior work has documented conversational features correlated with sycophancy, we lack a systematic understanding of what provokes or prevents AI sycophancy. Here, we present a set of controlled experimental studies where we first isolate how input framing influences sycophancy, and second, leverage these findings to develop mitigation strategies. In a nested factorial design, we compare questions to various non-questions where we vary three orthogonal factors: epistemic certainty (statement, belief, conviction), perspective (I- vs user-perspective), and affirmation vs negation. We show that (1) sycophancy is substantially higher in response to non-questions compared to questions. Additionally, we find that (2) sycophancy increases monotonically with epistemic certainty conveyed by the user, and (3) is amplified by I-perspective framing. Building on this, we show that asking a model to convert non-questions into questions before answering significantly reduces sycophancy. Importantly, this effect is stronger than a simple baseline prompt asking models "not to be sycophantic". Our work offers a practical and effective input-level mitigation that both developers and users can easily adopt.