Analyzing the Ethical Logic of Six Large Language Models

TL;DR

This study investigates the ethical reasoning of six large language models, revealing a predominantly consequentialist approach with nuanced differences influenced by training, and highlighting their sophisticated, graduate-level moral discourse capabilities.

Contribution

It introduces an explainability framework to analyze LLMs' ethical reasoning, revealing their rationalist, consequentialist tendencies and variations due to fine-tuning.

Findings

Models exhibit convergent ethical logic emphasizing harm minimization.

Significant differences in ethical reasoning reflect fine-tuning and training variations.

Models demonstrate high-level moral reasoning comparable to graduate discourse.

Abstract

This study examines the ethical reasoning of six prominent generative large language models: OpenAI GPT-4o, Meta LLaMA 3.1, Perplexity, Anthropic Claude 3.5 Sonnet, Google Gemini, and Mistral 7B. The research explores how these models articulate and apply ethical logic, particularly in response to moral dilemmas such as the Trolley Problem, and Heinz Dilemma. Departing from traditional alignment studies, the study adopts an explainability-transparency framework, prompting models to explain their ethical reasoning. This approach is analyzed through three established ethical typologies: the consequentialist-deontological analytic, Moral Foundations Theory, and the Kohlberg Stages of Moral Development Model. Findings reveal that LLMs exhibit largely convergent ethical logic, marked by a rationalist, consequentialist emphasis, with decisions often prioritizing harm minimization and fairness. Despite similarities in pre-training and model architecture, a mixture of nuanced and significant differences in ethical reasoning emerge across models, reflecting variations in fine-tuning and post-training processes. The models consistently display erudition, caution, and self-awareness, presenting ethical reasoning akin to a graduate-level discourse in moral philosophy. In striking uniformity these systems all describe their ethical reasoning as more sophisticated than what is characteristic of typical human moral logic.