Learning Transferable Concepts in Deep Reinforcement Learning

TL;DR

This paper proposes a self-supervised, information-theoretic approach to learning discrete, high-level representations in deep reinforcement learning, enabling transferability of concepts across tasks and improving sample efficiency.

Contribution

It introduces a novel method for learning transferable concepts via discrete representations, enhancing generalization in deep reinforcement learning.

Findings

Improved sample efficiency in locomotive and control tasks

Successful transfer of learned concepts to new tasks

Demonstrated generalization abilities in reinforcement learning agents

Abstract

While humans and animals learn incrementally during their lifetimes and exploit their experience to solve new tasks, standard deep reinforcement learning methods specialize to solve only one task at a time. As a result, the information they acquire is hardly reusable in new situations. Here, we introduce a new perspective on the problem of leveraging prior knowledge to solve future tasks. We show that learning discrete representations of sensory inputs can provide a high-level abstraction that is common across multiple tasks, thus facilitating the transference of information. In particular, we show that it is possible to learn such representations by self-supervision, following an information theoretic approach. Our method is able to learn concepts in locomotive and optimal control tasks that increase the sample efficiency in both known and unknown tasks, opening a new path to endow artificial agents with generalization abilities.