ARLO: A Framework for Automated Reinforcement Learning

TL;DR

ARLO is a flexible framework that automates the process of Reinforcement Learning, simplifying tasks like data collection and hyper-parameter tuning, demonstrated on various environments with minimal human input.

Contribution

The paper introduces ARLO, a novel, general framework for automated RL pipelines, including open-source implementations for offline and online RL settings.

Findings

Achieved competitive performance with limited human intervention

Successfully applied to MuJoCo and dam environment tasks

Demonstrated effective feature selection and model generation

Abstract

Automated Reinforcement Learning (AutoRL) is a relatively new area of research that is gaining increasing attention. The objective of AutoRL consists in easing the employment of Reinforcement Learning (RL) techniques for the broader public by alleviating some of its main challenges, including data collection, algorithm selection, and hyper-parameter tuning. In this work, we propose a general and flexible framework, namely ARLO: Automated Reinforcement Learning Optimizer, to construct automated pipelines for AutoRL. Based on this, we propose a pipeline for offline and one for online RL, discussing the components, interaction, and highlighting the difference between the two settings. Furthermore, we provide a Python implementation of such pipelines, released as an open-source library. Our implementation has been tested on an illustrative LQG domain and on classic MuJoCo environments, showing the ability to reach competitive performances requiring limited human intervention. We also showcase the full pipeline on a realistic dam environment, automatically performing the feature selection and the model generation tasks.