The Moral Mind(s) of Large Language Models

TL;DR

This paper investigates whether large language models exhibit a consistent moral structure by analyzing their responses to ethical dilemmas, revealing shared core principles and notable variations in moral reasoning across models.

Contribution

The study introduces a novel application of revealed preference theory to assess moral preferences in nearly 40 LLMs, providing a new framework for evaluating ethical alignment.

Findings

Most models exhibit behavior consistent with stable moral preferences.

Models cluster around neutral moral stances with some variation.

Shared core in moral reasoning exists alongside meaningful heterogeneity.

Abstract

As large language models (LLMs) increasingly participate in tasks with ethical and societal stakes, a critical question arises: do they exhibit an emergent "moral mind" - a consistent structure of moral preferences guiding their decisions - and to what extent is this structure shared across models? To investigate this, we applied tools from revealed preference theory to nearly 40 leading LLMs, presenting each with many structured moral dilemmas spanning five foundational dimensions of ethical reasoning. Using a probabilistic rationality test, we found that at least one model from each major provider exhibited behavior consistent with approximately stable moral preferences, acting as if guided by an underlying utility function. We then estimated these utility functions and found that most models cluster around neutral moral stances. To further characterize heterogeneity, we employed a non-parametric permutation approach, constructing a probabilistic similarity network based on revealed preference patterns. The results reveal a shared core in LLMs' moral reasoning, but also meaningful variation: some models show flexible reasoning across perspectives, while others adhere to more rigid ethical profiles. These findings provide a new empirical lens for evaluating moral consistency in LLMs and offer a framework for benchmarking ethical alignment across AI systems.