Acting for the Right Reasons: Creating Reason-Sensitive Artificial Moral Agents

TL;DR

This paper introduces a reinforcement learning framework that incorporates normative reasons to enable moral decision-making in artificial agents, using a reason-based shield generator and iterative improvement via moral feedback.

Contribution

It presents a novel architecture integrating normative reasons into reinforcement learning and an algorithm for refining moral decision-making through case-based feedback.

Findings

The reason-based shield effectively constrains agents to morally justified actions.

The iterative algorithm improves the moral reasoning of agents over time.

The approach aligns AI behavior with recognized normative principles.

Abstract

We propose an extension of the reinforcement learning architecture that enables moral decision-making of reinforcement learning agents based on normative reasons. Central to this approach is a reason-based shield generator yielding a moral shield that binds the agent to actions that conform with recognized normative reasons so that our overall architecture restricts the agent to actions that are (internally) morally justified. In addition, we describe an algorithm that allows to iteratively improve the reason-based shield generator through case-based feedback from a moral judge.