Decoupling Dynamics and Reward for Transfer Learning

TL;DR

This paper introduces a decoupled learning approach in reinforcement learning that separates task components to improve transferability and robustness across different domains and changes.

Contribution

It proposes a novel decoupled learning framework that isolates task representation, dynamics, and reward, enhancing transfer and online planning capabilities.

Findings

Improved transfer performance across domain perturbations

Effective application in both continuous and discrete RL tasks

Enhanced online planning through shared representation

Abstract

Current reinforcement learning (RL) methods can successfully learn single tasks but often generalize poorly to modest perturbations in task domain or training procedure. In this work, we present a decoupled learning strategy for RL that creates a shared representation space where knowledge can be robustly transferred. We separate learning the task representation, the forward dynamics, the inverse dynamics and the reward function of the domain, and show that this decoupling improves performance within the task, transfers well to changes in dynamics and reward, and can be effectively used for online planning. Empirical results show good performance in both continuous and discrete RL domains.