Aligning AI With Shared Human Values

TL;DR

This paper introduces the ETHICS dataset to evaluate language models' understanding of human morality, highlighting current capabilities and gaps in predicting ethical judgments, and aims to guide the development of AI aligned with human values.

Contribution

The paper presents the ETHICS dataset as a new benchmark for assessing language models' moral knowledge and demonstrates their current partial understanding of human ethics.

Findings

Models can predict some moral judgments but are incomplete.

Progress is possible in machine ethics with current models.

The dataset enables future improvements in AI value alignment.

Abstract

We show how to assess a language model's knowledge of basic concepts of morality. We introduce the ETHICS dataset, a new benchmark that spans concepts in justice, well-being, duties, virtues, and commonsense morality. Models predict widespread moral judgments about diverse text scenarios. This requires connecting physical and social world knowledge to value judgements, a capability that may enable us to steer chatbot outputs or eventually regularize open-ended reinforcement learning agents. With the ETHICS dataset, we find that current language models have a promising but incomplete ability to predict basic human ethical judgements. Our work shows that progress can be made on machine ethics today, and it provides a steppingstone toward AI that is aligned with human values.