Transfer Learning in Deep Reinforcement Learning: A Survey

TL;DR

This survey reviews recent advances in transfer learning within deep reinforcement learning, highlighting how knowledge transfer improves learning efficiency and effectiveness across various applications and identifying future research challenges.

Contribution

It provides a comprehensive framework categorizing state-of-the-art transfer learning methods in deep reinforcement learning and analyzes their goals, methodologies, and applications.

Findings

Categorization framework for transfer learning approaches

Analysis of transfer learning goals and methodologies

Identification of future research challenges

Abstract

Reinforcement learning is a learning paradigm for solving sequential decision-making problems. Recent years have witnessed remarkable progress in reinforcement learning upon the fast development of deep neural networks. Along with the promising prospects of reinforcement learning in numerous domains such as robotics and game-playing, transfer learning has arisen to tackle various challenges faced by reinforcement learning, by transferring knowledge from external expertise to facilitate the efficiency and effectiveness of the learning process. In this survey, we systematically investigate the recent progress of transfer learning approaches in the context of deep reinforcement learning. Specifically, we provide a framework for categorizing the state-of-the-art transfer learning approaches, under which we analyze their goals, methodologies, compatible reinforcement learning backbones, and practical applications. We also draw connections between transfer learning and other relevant topics from the reinforcement learning perspective and explore their potential challenges that await future research progress.