Environments for Lifelong Reinforcement Learning

TL;DR

This paper discusses the characteristics of environments suitable for training and evaluating lifelong reinforcement learning agents, reviews existing environments, and offers recommendations for future development to support continuous learning.

Contribution

It provides a comprehensive analysis of environments for lifelong RL and proposes guidelines for designing future environments to facilitate continuous learning.

Findings

Existing environments often lack support for lifelong learning

Recommendations for environment design to promote continual skill acquisition

Analysis of how current environments meet or fall short of lifelong learning needs

Abstract

To achieve general artificial intelligence, reinforcement learning (RL) agents should learn not only to optimize returns for one specific task but also to constantly build more complex skills and scaffold their knowledge about the world, without forgetting what has already been learned. In this paper, we discuss the desired characteristics of environments that can support the training and evaluation of lifelong reinforcement learning agents, review existing environments from this perspective, and propose recommendations for devising suitable environments in the future.