Reward Dimension Reduction for Scalable Multi-Objective Reinforcement Learning

TL;DR

This paper presents a reward dimension reduction technique for multi-objective reinforcement learning that improves scalability and efficiency, especially in environments with many objectives, while maintaining Pareto-optimality.

Contribution

The paper introduces an online reward dimension reduction method tailored for multi-objective reinforcement learning, capable of handling many objectives and preserving Pareto-optimality.

Findings

Outperforms existing methods in environments with up to sixteen objectives.

Enhances learning efficiency and policy performance in multi-objective settings.

Demonstrates effectiveness through a new training and evaluation framework.

Abstract

In this paper, we introduce a simple yet effective reward dimension reduction method to tackle the scalability challenges of multi-objective reinforcement learning algorithms. While most existing approaches focus on optimizing two to four objectives, their abilities to scale to environments with more objectives remain uncertain. Our method uses a dimension reduction approach to enhance learning efficiency and policy performance in multi-objective settings. While most traditional dimension reduction methods are designed for static datasets, our approach is tailored for online learning and preserves Pareto-optimality after transformation. We propose a new training and evaluation framework for reward dimension reduction in multi-objective reinforcement learning and demonstrate the superiority of our method in environments including one with sixteen objectives, significantly outperforming existing online dimension reduction methods.