Learning Abstract and Transferable Representations for Planning

TL;DR

This paper introduces a framework for learning task-independent, transferable state abstractions for long-term planning, enabling agents to reuse knowledge across different environments and tasks, thereby improving sample efficiency.

Contribution

The paper presents a novel method for autonomously learning hierarchical, transferable state abstractions from sensory data for long-term planning tasks.

Findings

Transferable abstractions improve planning efficiency.

Hierarchical representations enable reuse in new environments.

Sample efficiency increases with more tasks learned.

Abstract

We are concerned with the question of how an agent can acquire its own representations from sensory data. We restrict our focus to learning representations for long-term planning, a class of problems that state-of-the-art learning methods are unable to solve. We propose a framework for autonomously learning state abstractions of an agent's environment, given a set of skills. Importantly, these abstractions are task-independent, and so can be reused to solve new tasks. We demonstrate how an agent can use an existing set of options to acquire representations from ego- and object-centric observations. These abstractions can immediately be reused by the same agent in new environments. We show how to combine these portable representations with problem-specific ones to generate a sound description of a specific task that can be used for abstract planning. Finally, we show how to autonomously construct a multi-level hierarchy consisting of increasingly abstract representations. Since these hierarchies are transferable, higher-order concepts can be reused in new tasks, relieving the agent from relearning them and improving sample efficiency. Our results demonstrate that our approach allows an agent to transfer previous knowledge to new tasks, improving sample efficiency as the number of tasks increases.