Advancements and Challenges in Continual Reinforcement Learning: A Comprehensive Review

TL;DR

This comprehensive review explores recent progress, core challenges, and future directions in continual reinforcement learning, emphasizing its applications in robotics and the importance of evaluation environments for advancing the field.

Contribution

It provides an extensive overview of fundamental concepts, recent advancements, and evaluation methods in continual reinforcement learning, highlighting challenges and future research directions.

Findings

Recent advancements in continual RL within robotics

Identification of key challenges in continual RL

Overview of evaluation environments used in research

Abstract

The diversity of tasks and dynamic nature of reinforcement learning (RL) require RL agents to be able to learn sequentially and continuously, a learning paradigm known as continuous reinforcement learning. This survey reviews how continual learning transforms RL agents into dynamic continual learners. This enables RL agents to acquire and retain useful and reusable knowledge seamlessly. The paper delves into fundamental aspects of continual reinforcement learning, exploring key concepts, significant challenges, and novel methodologies. Special emphasis is placed on recent advancements in continual reinforcement learning within robotics, along with a succinct overview of evaluation environments utilized in prominent research, facilitating accessibility for newcomers to the field. The review concludes with a discussion on limitations and promising future directions, providing valuable insights for researchers and practitioners alike.