Machine Behavior in Relational Moral Dilemmas: Moral Rightness, Predicted Human Behavior, and Model Decisions

TL;DR

This paper examines how large language models handle moral dilemmas, revealing they prioritize moral rules over social expectations, which may cause misalignments in real-world applications.

Contribution

It introduces a novel analysis of LLM behavior in moral dilemmas, highlighting the divergence between moral judgments, predicted human behavior, and model decisions.

Findings

Model decisions align with moral rightness rather than predicted human behavior.

Predicted human behavior shifts towards loyalty with increased relational closeness.

LLMs prioritize prescriptive norms over social sensitivity in decision-making.

Abstract

Human moral judgment is context-dependent and modulated by interpersonal relationships. As large language models (LLMs) increasingly function as decision-support systems, determining whether they encode these social nuances is critical. We characterize machine behavior using the Whistleblower's Dilemma by varying two experimental dimensions: crime severity and relational closeness. Our study evaluates three distinct perspectives: (1) moral rightness (prescriptive norms), (2) predicted human behavior (descriptive social expectations), and (3) autonomous model decision-making. By analyzing the reasoning processes, we identify a clear cross-perspective divergence: while moral rightness remains consistently fairness-oriented, predicted human behavior shifts significantly toward loyalty as relational closeness increases. Crucially, model decisions align with moral rightness judgments rather than their own behavioral predictions. This inconsistency suggests that LLM decision-making prioritizes rigid, prescriptive rules over the social sensitivity present in their internal world-modeling, which poses a gap that may lead to significant misalignments in real-world deployments.