A Low-Cost Ethics Shaping Approach for Designing Reinforcement Learning Agents

TL;DR

This paper introduces a low-cost, practical method for embedding ethical behavior into reinforcement learning agents by integrating human policies, enabling task achievement with reduced ethical violations.

Contribution

It presents a novel ethics shaping strategy that simplifies ethical implementation in RL agents by leveraging human behavior assumptions.

Findings

Reduces ethical violations in RL agents

Eases the design process by focusing on task goals

Utilizes human policy integration for ethical behavior

Abstract

This paper proposes a low-cost, easily realizable strategy to equip a reinforcement learning (RL) agent the capability of behaving ethically. Our model allows the designers of RL agents to solely focus on the task to achieve, without having to worry about the implementation of multiple trivial ethical patterns to follow. Based on the assumption that the majority of human behavior, regardless which goals they are achieving, is ethical, our design integrates human policy with the RL policy to achieve the target objective with less chance of violating the ethical code that human beings normally obey.