A Continual Offline Reinforcement Learning Benchmark for Navigation Tasks

TL;DR

This paper introduces a comprehensive benchmark suite for continual offline reinforcement learning in navigation tasks, addressing challenges like catastrophic forgetting, task adaptation, and scalability to advance research and practical applications.

Contribution

The paper presents a new benchmark with diverse video-game navigation scenarios, evaluation protocols, and baseline comparisons to facilitate reproducible research in continual reinforcement learning.

Findings

Benchmark effectively captures key challenges in continual RL

Baseline algorithms show varied performance across tasks

Framework supports reproducibility and practical deployment

Abstract

Autonomous agents operating in domains such as robotics or video game simulations must adapt to changing tasks without forgetting about the previous ones. This process called Continual Reinforcement Learning poses non-trivial difficulties, from preventing catastrophic forgetting to ensuring the scalability of the approaches considered. Building on recent advances, we introduce a benchmark providing a suite of video-game navigation scenarios, thus filling a gap in the literature and capturing key challenges : catastrophic forgetting, task adaptation, and memory efficiency. We define a set of various tasks and datasets, evaluation protocols, and metrics to assess the performance of algorithms, including state-of-the-art baselines. Our benchmark is designed not only to foster reproducible research and to accelerate progress in continual reinforcement learning for gaming, but also to provide a reproducible framework for production pipelines -- helping practitioners to identify and to apply effective approaches.