Avalanche RL: a Continual Reinforcement Learning Library

TL;DR

Avalanche RL is a flexible, PyTorch-based library designed for continual reinforcement learning, supporting various environments and facilitating research through a new benchmark and integration with Habitat-Sim.

Contribution

It introduces Avalanche RL, a comprehensive library for CRL, and proposes Continual Habitat-Lab, a new benchmark using photorealistic simulation for advancing CRL research.

Findings

Supports any OpenAI Gym environment

Enables easy training on continuous task streams

Facilitates research with a new photorealistic benchmark

Abstract

Continual Reinforcement Learning (CRL) is a challenging setting where an agent learns to interact with an environment that is constantly changing over time (the stream of experiences). In this paper, we describe Avalanche RL, a library for Continual Reinforcement Learning which allows to easily train agents on a continuous stream of tasks. Avalanche RL is based on PyTorch and supports any OpenAI Gym environment. Its design is based on Avalanche, one of the more popular continual learning libraries, which allow us to reuse a large number of continual learning strategies and improve the interaction between reinforcement learning and continual learning researchers. Additionally, we propose Continual Habitat-Lab, a novel benchmark and a high-level library which enables the usage of the photorealistic simulator Habitat-Sim for CRL research. Overall, Avalanche RL attempts to unify under a common framework continual reinforcement learning applications, which we hope will foster the growth of the field.