Object-oriented state editing for HRL

TL;DR

This paper presents an object-oriented hierarchical reinforcement learning approach where a controller manipulates scene objects to efficiently explore alternative states, improving data efficiency while maintaining performance.

Contribution

It introduces a novel hierarchical controller that performs object-oriented state editing using graph-based scene representations in HRL.

Findings

Achieves similar rewards as non-hierarchical agents

Demonstrates improved data efficiency

Validates approach on three environments

Abstract

We introduce agents that use object-oriented reasoning to consider alternate states of the world in order to more quickly find solutions to problems. Specifically, a hierarchical controller directs a low-level agent to behave as if objects in the scene were added, deleted, or modified. The actions taken by the controller are defined over a graph-based representation of the scene, with actions corresponding to adding, deleting, or editing the nodes of a graph. We present preliminary results on three environments, demonstrating that our approach can achieve similar levels of reward as non-hierarchical agents, but with better data efficiency.