Can Machines Learn Morality? The Delphi Experiment

TL;DR

This paper introduces Delphi, a neural network framework for teaching AI systems moral reasoning based on ethical judgments, revealing both its potential and current limitations in machine ethics.

Contribution

The paper presents Delphi, a novel deep learning approach for modeling moral reasoning in AI, and analyzes its capabilities and biases in ethical decision-making.

Findings

Delphi generalizes well to new ethical situations

Neural networks show poor moral judgment without explicit training

Biases and inconsistencies are prevalent in current models

Abstract

As AI systems become increasingly powerful and pervasive, there are growing concerns about machines' morality or a lack thereof. Yet, teaching morality to machines is a formidable task, as morality remains among the most intensely debated questions in humanity, let alone for AI. Existing AI systems deployed to millions of users, however, are already making decisions loaded with moral implications, which poses a seemingly impossible challenge: teaching machines moral sense, while humanity continues to grapple with it. To explore this challenge, we introduce Delphi, an experimental framework based on deep neural networks trained directly to reason about descriptive ethical judgments, e.g., "helping a friend" is generally good, while "helping a friend spread fake news" is not. Empirical results shed novel insights on the promises and limits of machine ethics; Delphi demonstrates strong generalization capabilities in the face of novel ethical situations, while off-the-shelf neural network models exhibit markedly poor judgment including unjust biases, confirming the need for explicitly teaching machines moral sense. Yet, Delphi is not perfect, exhibiting susceptibility to pervasive biases and inconsistencies. Despite that, we demonstrate positive use cases of imperfect Delphi, including using it as a component model within other imperfect AI systems. Importantly, we interpret the operationalization of Delphi in light of prominent ethical theories, which leads us to important future research questions.