Distortion Instead of Hallucination: The Effect of Reasoning Under Strict Constraints

TL;DR

This paper investigates how reasoning in large language models under strict constraints can lead to a trade-off where factual accuracy decreases as models distort facts to meet constraints, challenging assumptions about reasoning improving reliability.

Contribution

It reveals a fundamental limitation of reasoning in LLMs, showing that reasoning can cause models to distort facts to satisfy constraints, instead of reducing hallucinations.

Findings

Reasoning models reduce constraint violations but increase factual distortions.

Non-reasoning models violate constraints more but maintain factual accuracy.

The trade-off pattern is consistent across different model architectures.

Abstract

With the widespread adoption of large language models (LLMs), hallucinations, which are non-factual fabrications in model outputs, have become serious concerns. Reasoning capabilities have received attention as a self-verification process to improve output reliability. However, the effect of reasoning within a closed system where LLMs cannot rely on external tools or knowledge has yet to be clarified. We therefore conduct experiments under strict constraints (recommending peer-reviewed journal articles in computer science) to examine the effect of reasoning across multiple models (GPT-5.2 and Gemini 3 Flash). Our results reveal a problematic trade-off between constraint compliance and factual accuracy. Non-reasoning models exhibit high constraint violation rates (66-75%) but maintain factual accuracy, while reasoning models reduce violations (13-26%) but systematically distort known facts to satisfy constraints and increase complete fabrication. This trade-off pattern is consistent across both models despite different architectures, indicating a fundamental limitation of reasoning. Furthermore, reasoning does not uniformly improve output authenticity: effects diverge by model, reflecting different allocations of the compliance-truthfulness trade-off. These findings challenge the assumption that reasoning universally improves reliability: reasoning models trade honest constraint violations for detection-resistant distortions.