The Alignment Target Problem: Divergent Moral Judgments of Humans, AI Systems, and Their Designers

TL;DR

This study investigates how moral judgments differ when evaluating humans, AI systems, and their designers, revealing that visibility of human agency influences moral evaluations and complicates the alignment of AI behavior with human values.

Contribution

It extends the alignment target problem by empirically showing that moral evaluations vary based on perceived agency and origin of AI actions, challenging existing assumptions.

Findings

Judgments of robots and humans are similar when actions are anonymous.

Judgments shift to rule-based reasoning when AI is described as human-designed.

People evaluate human designers and AI systems differently in moral scenarios.

Abstract

The project of aligning machine behavior with human values raises a basic problem: whose moral expectations should guide AI decision-making? Much alignment research assumes that the appropriate benchmark is how humans themselves would act in a given situation. Studies of agent-type value forks challenge this assumption by showing that people do not always judge humans and AI systems identically.This paper extends that challenge by examining two further possibilities: first, that evaluations of AI behavior change when its human origins are made visible; and second, that people judge the humans who program AI systems differently from either the machines or the human actors they are compared against. An experiment with 1,002 U.S. adults measured moral judgments in a runaway mine train scenario, varying the subject of evaluation across four conditions: a repairman, a repair robot, a repair robot programmed by company engineers, and company engineers programming a repair robot. We find no significant difference in evaluations of the repairman and the robot. However, judgments shifted substantially when the robot's actions were described as the product of human design. Participants exhibited markedly more deontological, rule-based reasoning when evaluating either the programmed robot or the engineers who programmed it, suggesting that rendering human agency visible activates heightened moral constraints. These findings indicate that people may evaluate humans, AI systems acting in the same situation, and the humans who design them in meaningfully different ways. The fact that these evaluations do not necessarily converge gives rise to the alignment target problem: which normative target should guide the development of artificial moral agents in high-stakes domains, and whether these plural judgments can be reconciled within a coherent account of value alignment.