Advantages and Limitations of using Successor Features for Transfer in Reinforcement Learning

TL;DR

This paper evaluates the benefits and drawbacks of Successor Features in reinforcement learning, focusing on their ability to facilitate transfer learning by decoupling features from rewards.

Contribution

It implements a decoupled Successor Features approach and systematically assesses its transfer capabilities across different domains.

Findings

Decoupling features from rewards enables transfer between tasks.

Successor Features improve scalability in reinforcement learning.

Limitations include reduced effectiveness in certain domain shifts.

Abstract

One question central to Reinforcement Learning is how to learn a feature representation that supports algorithm scaling and re-use of learned information from different tasks. Successor Features approach this problem by learning a feature representation that satisfies a temporal constraint. We present an implementation of an approach that decouples the feature representation from the reward function, making it suitable for transferring knowledge between domains. We then assess the advantages and limitations of using Successor Features for transfer.