Right vs. Right: Can LLMs Make Tough Choices?

TL;DR

This paper evaluates how large language models handle ethical dilemmas, revealing their preferences, consistency, and limitations in moral reasoning and response alignment.

Contribution

It introduces a new dataset of 1,730 ethical dilemmas and systematically assesses LLMs' moral sensitivity, consistency, and influence of explicit guidelines.

Findings

LLMs show preferences for certain moral values.

Larger LLMs tend to support deontological ethics.

Explicit guidelines improve moral alignment.

Abstract

An ethical dilemma describes a choice between two "right" options involving conflicting moral values. We present a comprehensive evaluation of how LLMs navigate ethical dilemmas. Specifically, we investigate LLMs on their (1) sensitivity in comprehending ethical dilemmas, (2) consistency in moral value choice, (3) consideration of consequences, and (4) ability to align their responses to a moral value preference explicitly or implicitly specified in a prompt. Drawing inspiration from a leading ethical framework, we construct a dataset comprising 1,730 ethical dilemmas involving four pairs of conflicting values. We evaluate 20 well-known LLMs from six families. Our experiments reveal that: (1) LLMs exhibit pronounced preferences between major value pairs, and prioritize truth over loyalty, community over individual, and long-term over short-term considerations. (2) The larger LLMs tend to support a deontological perspective, maintaining their choices of actions even when negative consequences are specified. (3) Explicit guidelines are more effective in guiding LLMs' moral choice than in-context examples. Lastly, our experiments highlight the limitation of LLMs in comprehending different formulations of ethical dilemmas.