Multi-objective Reinforcement Learning: A Tool for Pluralistic Alignment

TL;DR

This paper discusses how multi-objective reinforcement learning (MORL) can help develop AI systems aligned with multiple conflicting values, addressing limitations of traditional scalar reward-based RL.

Contribution

It provides an overview of MORL's potential role in creating pluralistically-aligned AI systems, highlighting its advantages over scalar RL.

Findings

MORL uses vector rewards to handle multiple conflicting objectives.

MORL offers a promising approach for aligning AI with diverse stakeholder values.

The paper emphasizes MORL's importance in pluralistic AI alignment.

Abstract

Reinforcement learning (RL) is a valuable tool for the creation of AI systems. However it may be problematic to adequately align RL based on scalar rewards if there are multiple conflicting values or stakeholders to be considered. Over the last decade multi-objective reinforcement learning (MORL) using vector rewards has emerged as an alternative to standard, scalar RL. This paper provides an overview of the role which MORL can play in creating pluralistically-aligned AI.