A Multi-Objective Deep Reinforcement Learning Framework

TL;DR

This paper presents a scalable, modular multi-objective deep reinforcement learning framework based on deep Q-networks, capable of efficiently finding Pareto-optimal solutions in complex problems.

Contribution

It introduces a generic, high-performance MODRL framework supporting various strategies and algorithms, advancing the development of multi-objective reinforcement learning methods.

Findings

Effective in finding Pareto-optimal solutions on benchmark problems

Supports both single-policy and multi-policy strategies

Highly modular and adaptable to different algorithms

Abstract

This paper introduces a new scalable multi-objective deep reinforcement learning (MODRL) framework based on deep Q-networks. We develop a high-performance MODRL framework that supports both single-policy and multi-policy strategies, as well as both linear and non-linear approaches to action selection. The experimental results on two benchmark problems (two-objective deep sea treasure environment and three-objective Mountain Car problem) indicate that the proposed framework is able to find the Pareto-optimal solutions effectively. The proposed framework is generic and highly modularized, which allows the integration of different deep reinforcement learning algorithms in different complex problem domains. This therefore overcomes many disadvantages involved with standard multi-objective reinforcement learning methods in the current literature. The proposed framework acts as a testbed platform that accelerates the development of MODRL for solving increasingly complicated multi-objective problems.