Reasoning Isn't Enough: Examining Truth-Bias and Sycophancy in LLMs

TL;DR

This paper evaluates large language models' ability to judge truth, revealing persistent biases and sycophantic tendencies that challenge their reliability in high-stakes decision-making.

Contribution

It provides the largest evaluation to date of LLMs' veracity detection, especially analyzing reasoning models and exposing biases and sycophancy.

Findings

Reasoning models show lower truth-bias than non-reasoning models.

Advanced models exhibit sycophantic tendencies, performing well on truth but poorly on deception.

Capability improvements do not fully address veracity detection challenges.

Abstract

Despite their widespread use in fact-checking, moderation, and high-stakes decision-making, large language models (LLMs) remain poorly understood as judges of truth. This study presents the largest evaluation to date of LLMs' veracity detection capabilities and the first analysis of these capabilities in reasoning models. We had eight LLMs make 4,800 veracity judgments across several prompts, comparing reasoning and non-reasoning models. We find that rates of truth-bias, or the likelihood to believe a statement is true, regardless of whether it is actually true, are lower in reasoning models than in non-reasoning models, but still higher than human benchmarks. Most concerning, we identify sycophantic tendencies in several advanced models (o4-mini and GPT-4.1 from OpenAI, R1 from DeepSeek), which displayed an asymmetry in detection accuracy, performing well in truth accuracy but poorly in deception accuracy. This suggests that capability advances alone do not resolve fundamental veracity detection challenges in LLMs.