Multi-Source Transfer Learning for Deep Model-Based Reinforcement Learning

TL;DR

This paper introduces modular multi-source transfer learning techniques to improve reinforcement learning by automatically selecting and transferring knowledge from multiple source tasks, even across different domains.

Contribution

It presents novel methods that automatically determine what knowledge to transfer from multiple sources, addressing key challenges in transfer learning for reinforcement learning.

Findings

Effective transfer across different domains demonstrated

Reduces environment interactions needed for learning

Improves performance in visual control tasks

Abstract

A crucial challenge in reinforcement learning is to reduce the number of interactions with the environment that an agent requires to master a given task. Transfer learning proposes to address this issue by re-using knowledge from previously learned tasks. However, determining which source task qualifies as the most appropriate for knowledge extraction, as well as the choice regarding which algorithm components to transfer, represent severe obstacles to its application in reinforcement learning. The goal of this paper is to address these issues with modular multi-source transfer learning techniques. The proposed techniques automatically learn how to extract useful information from source tasks, regardless of the difference in state-action space and reward function. We support our claims with extensive and challenging cross-domain experiments for visual control.