Combined Reinforcement Learning via Abstract Representations

TL;DR

This paper introduces a novel reinforcement learning framework that combines model-free and model-based methods through a shared low-dimensional environment encoding, enhancing generalization, efficiency, and interpretability.

Contribution

It proposes a new approach that explicitly bridges model-free and model-based reinforcement learning using a shared abstract environment representation.

Findings

Modular approach improves generalization and efficiency.

Planning occurs in a smaller latent space, reducing computational costs.

Recovered low-dimensional environment representations enable interpretability and transfer learning.

Abstract

In the quest for efficient and robust reinforcement learning methods, both model-free and model-based approaches offer advantages. In this paper we propose a new way of explicitly bridging both approaches via a shared low-dimensional learned encoding of the environment, meant to capture summarizing abstractions. We show that the modularity brought by this approach leads to good generalization while being computationally efficient, with planning happening in a smaller latent state space. In addition, this approach recovers a sufficient low-dimensional representation of the environment, which opens up new strategies for interpretable AI, exploration and transfer learning.