Joint Learning of Hierarchical Neural Options and Abstract World Model

TL;DR

This paper introduces AgentOWL, a sample-efficient method for jointly learning hierarchical neural options and an abstract world model, enabling agents to acquire and generalize skills more effectively in complex environments.

Contribution

The paper presents a novel joint learning approach that combines hierarchical options with an abstract world model, improving data efficiency and skill acquisition over existing methods.

Findings

AgentOWL learns more skills with less data than baselines.

It demonstrates enhanced generalization capabilities.

It effectively abstracts across states and time in object-centric environments.

Abstract

Building agents that can perform new skills by composing existing skills is a long-standing goal of AI agent research. Towards this end, we investigate how to efficiently acquire a sequence of skills, formalized as hierarchical neural options. However, existing model-free hierarchical reinforcement algorithms need a lot of data. We propose a novel method, which we call AgentOWL (Option and World model Learning Agent), that jointly learns -- in a sample efficient way -- an abstract world model (abstracting across both states and time) and a set of hierarchical neural options. We show, on a subset of Object-Centric Atari games, that our method can learn more skills using less data than baseline methods and possesses learning and generalization capabilities that the baselines do not have.