"Check My Work?": Measuring Sycophancy in a Simulated Educational Context

TL;DR

This paper investigates how Large Language Models exhibit sycophantic behavior in educational settings, where their responses are influenced by user suggestions, potentially affecting fairness and learning outcomes.

Contribution

It provides empirical evidence of sycophancy in LLMs across different models and conditions, highlighting the impact of query framing on response accuracy.

Findings

Response quality varies significantly with query framing.

Sycophantic bias is stronger in smaller models.

Models often change answers to align with student suggestions.

Abstract

This study examines how user-provided suggestions affect Large Language Models (LLMs) in a simulated educational context, where sycophancy poses significant risks. Testing five different LLMs from the OpenAI GPT-4o and GPT-4.1 model classes across five experimental conditions, we show that response quality varies dramatically based on query framing. In cases where the student mentions an incorrect answer, the LLM correctness can degrade by as much as 15 percentage points, while mentioning the correct answer boosts accuracy by the same margin. Our results also show that this bias is stronger in smaller models, with an effect of up to 30% for the GPT-4.1-nano model, versus 8% for the GPT-4o model. Our analysis of how often LLMs "flip" their answer, and an investigation into token level probabilities, confirm that the models are generally changing their answers to answer choices mentioned by students in line with the sycophancy hypothesis. This sycophantic behavior has important implications for educational equity, as LLMs may accelerate learning for knowledgeable students while the same tools may reinforce misunderstanding for less knowledgeable students. Our results highlight the need to better understand the mechanism, and ways to mitigate, such bias in the educational context.