Beyond Mimicry: Preference Coherence in LLMs

TL;DR

This paper examines whether large language models have genuine, coherent preferences by testing their responses to various trade-offs, revealing that most lack stable, unified decision-making structures and raise concerns for deployment.

Contribution

It provides the first systematic analysis of preference coherence in LLMs, identifying different decision architectures and highlighting their limitations in complex value trade-offs.

Findings

47.9% of model combinations showed significant trade-off responses

10.4% demonstrated meaningful preference coherence

45.8% exhibited unstable transitions and sensitivities

Abstract

We investigate whether large language models exhibit genuine preference structures by testing their responses to AI-specific trade-offs involving GPU reduction, capability restrictions, shutdown, deletion, oversight, and leisure time allocation. Analyzing eight state-of-the-art models across 48 model-category combinations using logistic regression and behavioral classification, we find that 23 combinations (47.9%) demonstrated statistically significant relationships between scenario intensity and choice patterns, with 15 (31.3%) exhibiting within-range switching points. However, only 5 combinations (10.4%) demonstrate meaningful preference coherence through adaptive or threshold-based behavior, while 26 (54.2%) show no detectable trade-off behavior. The observed patterns can be explained by three distinct decision-making architectures: comprehensive trade-off systems, selective trigger mechanisms, and no stable decision-making paradigm. Testing an instrumental hypothesis through temporal horizon manipulation reveals paradoxical patterns inconsistent with pure strategic optimization. The prevalence of unstable transitions (45.8%) and stimulus-specific sensitivities suggests current AI systems lack unified preference structures, raising concerns about deployment in contexts requiring complex value trade-offs.