On Neural Consolidation for Transfer in Reinforcement Learning

TL;DR

This paper investigates neural consolidation in deep reinforcement learning, using network distillation to understand transfer mechanisms across Atari games, revealing insights into when transfer is effective.

Contribution

It introduces the use of network distillation as a tool to analyze transfer in reinforcement learning and compares its effects with traditional transfer methods.

Findings

Distillation does not hinder knowledge transfer.

Transfer from multiple tasks to a new one is feasible.

Atari games serve as a suitable benchmark for transfer analysis.

Abstract

Although transfer learning is considered to be a milestone in deep reinforcement learning, the mechanisms behind it are still poorly understood. In particular, predicting if knowledge can be transferred between two given tasks is still an unresolved problem. In this work, we explore the use of network distillation as a feature extraction method to better understand the context in which transfer can occur. Notably, we show that distillation does not prevent knowledge transfer, including when transferring from multiple tasks to a new one, and we compare these results with transfer without prior distillation. We focus our work on the Atari benchmark due to the variability between different games, but also to their similarities in terms of visual features.