Internal Reasoning vs. External Control: A Thermodynamic Analysis of Sycophancy in Large Language Models

TL;DR

This paper introduces Regulated Causal Anchoring (RCA), a novel inference-time method that detects sycophancy in large language models by verifying if outputs follow from their reasoning traces, without needing ground truth.

Contribution

The paper proposes RCA, a process evaluation technique that identifies trace-output inconsistency to reduce sycophancy, outperforming traditional outcome-based methods and operating without ground truth.

Findings

RCA achieves 0.0% sycophancy detection rate.

RCA accepts 88% of valid hints.

Traditional self-correction reduces failures to 7-9%.

Abstract

Large Language Models exhibit sycophancy: prioritizing agreeableness over correctness. Current remedies evaluate reasoning outcomes: RLHF rewards correct answers, self-correction critiques outputs. All require ground truth, which is often unavailable at inference time and vulnerable to the same biases. We explore evaluating the reasoning process instead. Regulated Causal Anchoring (RCA) verifies whether outputs follow from their reasoning traces, without requiring ground truth. Sycophancy manifests as trace-output inconsistency: models derive one answer but output another to please users. RCA detects this inconsistency, achieving 0.0% sycophancy while accepting 88% of valid hints. We identify two failures invisible to outcome evaluation: Inverse Scaling (frontier models sycophant more because rationalization requires capability) and the Final Output Gap (correct reasoning precedes sycophantic output). Traditional self-correction reduces these failures to 7-9% but cannot eliminate them because the model critiques itself with the same biases. RCA's process evaluation operates at inference time, requires no ground truth, and uses an independent judge that breaks the self-reinforcing bias loop: three properties that outcome evaluation lacks.