Zero-Shot Reinforcement Learning via Function Encoders

TL;DR

This paper introduces a function encoder for RL that enables zero-shot transfer by representing tasks as combinations of learned basis functions, improving transfer efficiency and stability.

Contribution

The paper proposes a novel function encoder method that allows zero-shot transfer in RL by encoding task functions as combinations of basis functions.

Findings

Achieves state-of-the-art data efficiency in RL tasks.

Demonstrates stable training and high asymptotic performance.

Enables zero-shot transfer across related tasks without additional training.

Abstract

Although reinforcement learning (RL) can solve many challenging sequential decision making problems, achieving zero-shot transfer across related tasks remains a challenge. The difficulty lies in finding a good representation for the current task so that the agent understands how it relates to previously seen tasks. To achieve zero-shot transfer, we introduce the function encoder, a representation learning algorithm which represents a function as a weighted combination of learned, non-linear basis functions. By using a function encoder to represent the reward function or the transition function, the agent has information on how the current task relates to previously seen tasks via a coherent vector representation. Thus, the agent is able to achieve transfer between related tasks at run time with no additional training. We demonstrate state-of-the-art data efficiency, asymptotic performance, and training stability in three RL fields by augmenting basic RL algorithms with a function encoder task representation.