DisCoRL: Continual Reinforcement Learning via Policy Distillation

TL;DR

DisCoRL introduces a method combining state representation learning and policy distillation to enable continual reinforcement learning, allowing agents to learn multiple tasks sequentially without forgetting and to infer the current task automatically.

Contribution

The paper presents DisCoRL, a novel approach that effectively addresses continual RL challenges by integrating state representation learning with policy distillation.

Findings

Successfully learned multiple navigation tasks sequentially

Automatically inferred the active task without external signals

Transferred policies from simulation to real-world robot

Abstract

In multi-task reinforcement learning there are two main challenges: at training time, the ability to learn different policies with a single model; at test time, inferring which of those policies applying without an external signal. In the case of continual reinforcement learning a third challenge arises: learning tasks sequentially without forgetting the previous ones. In this paper, we tackle these challenges by proposing DisCoRL, an approach combining state representation learning and policy distillation. We experiment on a sequence of three simulated 2D navigation tasks with a 3 wheel omni-directional robot. Moreover, we tested our approach's robustness by transferring the final policy into a real life setting. The policy can solve all tasks and automatically infer which one to run.