Cross-Domain Transfer in Reinforcement Learning using Target Apprentice

TL;DR

This paper introduces a transfer learning method in reinforcement learning that adapts and reuses source task policies for related domains, improving sample efficiency and generalization without re-learning from scratch.

Contribution

It proposes a novel approach to directly adapt source policies for cross-domain transfer in RL, accounting for model errors to enhance policy generalization.

Findings

Near-optimal policy transfer across related domains

Reduced sample complexity in target task learning

Improved sample efficiency through policy augmentation

Abstract

In this paper, we present a new approach to Transfer Learning (TL) in Reinforcement Learning (RL) for cross-domain tasks. Many of the available techniques approach the transfer architecture as a method of speeding up the target task learning. We propose to adapt and reuse the mapped source task optimal-policy directly in related domains. We show the optimal policy from a related source task can be near optimal in target domain provided an adaptive policy accounts for the model error between target and source. The main benefit of this policy augmentation is generalizing policies across multiple related domains without having to re-learn the new tasks. Our results show that this architecture leads to better sample efficiency in the transfer, reducing sample complexity of target task learning to target apprentice learning.