Grounding Hierarchical Reinforcement Learning Models for Knowledge Transfer

TL;DR

This paper introduces a deep model-based reinforcement learning approach that enables agents to learn and ground hierarchical representations of their environment through sensorimotor interactions, advancing the integration of deep learning with hierarchical RL.

Contribution

It proposes a novel deep hierarchical RL framework that allows for learning and grounding abstract representations, bridging the gap between model-free and model-based RL methods.

Findings

Hierarchical representations can be grounded through sensorimotor interaction.

The method enhances the abstraction capabilities of deep RL models.

Grounded hierarchies improve the understanding of complex environments.

Abstract

Methods of deep machine learning enable to to reuse low-level representations efficiently for generating more abstract high-level representations. Originally, deep learning has been applied passively (e.g., for classification purposes). Recently, it has been extended to estimate the value of actions for autonomous agents within the framework of reinforcement learning (RL). Explicit models of the environment can be learned to augment such a value function. Although "flat" connectionist methods have already been used for model-based RL, up to now, only model-free variants of RL have been equipped with methods from deep learning. We propose a variant of deep model-based RL that enables an agent to learn arbitrarily abstract hierarchical representations of its environment. In this paper, we present research on how such hierarchical representations can be grounded in sensorimotor interaction between an agent and its environment.