A Review of the Deep Sea Treasure problem as a Multi-Objective Reinforcement Learning Benchmark

TL;DR

This paper critically evaluates the Deep Sea Treasure problem as a multi-objective reinforcement learning benchmark, proposing an improved version that better reflects real-world complexities and providing comprehensive analysis and resources.

Contribution

The authors introduce an enhanced DST problem that addresses limitations of the original, along with theoretical proofs, a reference implementation, and a Pareto-front for benchmarking.

Findings

Original DST is overly simplistic and not representative of real problems.

The improved DST version has properties that differ from the original, making it more realistic.

Provided a reference implementation and Pareto-front for the new DST problem.

Abstract

In this paper, the authors investigate the Deep Sea Treasure (DST) problem as proposed by Vamplew et al. Through a number of proofs, the authors show the original DST problem to be quite basic, and not always representative of practical Multi-Objective Optimization problems. In an attempt to bring theory closer to practice, the authors propose an alternative, improved version of the DST problem, and prove that some of the properties that simplify the original DST problem no longer hold. The authors also provide a reference implementation and perform a comparison between their implementation, and other existing open-source implementations of the problem. Finally, the authors also provide a complete Pareto-front for their new DST problem.