DinerDash Gym: A Benchmark for Policy Learning in High-Dimensional Action Space

TL;DR

This paper introduces DinerDash Gym, a new benchmark for high-dimensional hierarchical policy learning, and proposes DPGM, a novel algorithm combining graph modeling and deep learning for improved performance.

Contribution

The paper presents a new benchmark with hierarchical structure and high-dimensional actions, and introduces DPGM, a method that embeds domain knowledge for better policy learning.

Findings

DPGM outperforms baseline algorithms in DinerDash Gym.

Domain knowledge injection improves policy learning.

DinerDash Gym facilitates research in complex hierarchical tasks.

Abstract

It has been arduous to assess the progress of a policy learning algorithm in the domain of hierarchical task with high dimensional action space due to the lack of a commonly accepted benchmark. In this work, we propose a new light-weight benchmark task called Diner Dash for evaluating the performance in a complicated task with high dimensional action space. In contrast to the traditional Atari games that only have a flat structure of goals and very few actions, the proposed benchmark task has a hierarchical task structure and size of 57 for the action space and hence can facilitate the development of policy learning in complicated tasks. On top of that, we introduce Decomposed Policy Graph Modelling (DPGM), an algorithm that combines both graph modelling and deep learning to allow explicit domain knowledge embedding and achieves significant improvement comparing to the baseline. In the experiments, we have shown the effectiveness of the domain knowledge injection via a specially designed imitation algorithm as well as results of other popular algorithms.