How Does Thinking Mode Change LLM Moral Judgments? A Controlled Instant-vs-Thinking Comparison Across Five Frontier Models

TL;DR

This study compares instant and reasoning modes in five frontier LLMs, finding reasoning narrows disagreements and alters ethical frameworks without affecting overall binary moral judgments.

Contribution

It provides the first systematic evaluation of how reasoning mode influences moral judgments across multiple advanced language models.

Findings

Reasoning narrows cross-model disagreement in disputed scenarios.

Reasoning reduces demographic-judgment inconsistency in most models.

Reasoning more often changes ethical frameworks than binary verdicts.

Abstract

We evaluate whether enabling provider-exposed reasoning mode changes moral judgments within the same model checkpoint. Across 100 moral-judgment scenarios and five frontier reasoning-trained LLMs (Claude Sonnet 4.6, GPT 5.5, Gemini 3 Flash, DeepSeek V3.1, and Qwen3.5 397B), aggregate binary-verdict agreement remains high and statistically indistinguishable between instant and thinking modes (Krippendorff's alpha = 0.78 vs. 0.79). However, disagreement is concentrated in 21 model-disputed scenarios, where instant-mode agreement is near chance (alpha = 0.08). On these scenarios, reasoning directionally narrows cross-model disagreement, increasing mean pairwise agreement from 5.4 to 6.7 out of 10. Reasoning also reduces demographic-judgment inconsistency in three of five models and does not increase it for any model. Across all five model families, reasoning changes self-labeled ethical frameworks more often than binary verdicts.