Gym4ReaL: A Suite for Benchmarking Real-World Reinforcement Learning

TL;DR

Gym4ReaL provides a diverse set of realistic environments to evaluate reinforcement learning algorithms under real-world challenges like large state spaces and partial observability, highlighting the need for new methods.

Contribution

Introduces Gym4ReaL, a comprehensive benchmark suite designed to evaluate RL algorithms in realistic, complex environments reflecting real-world challenges.

Findings

Standard RL algorithms are competitive with rule-based methods in these settings.

The benchmark exposes limitations of current RL methods in real-world scenarios.

Results motivate development of new algorithms for complex environments.

Abstract

In recent years, \emph{Reinforcement Learning} (RL) has made remarkable progress, achieving superhuman performance in a wide range of simulated environments. As research moves toward deploying RL in real-world applications, the field faces a new set of challenges inherent to real-world settings, such as large state-action spaces, non-stationarity, and partial observability. Despite their importance, these challenges are often underexplored in current benchmarks, which tend to focus on idealized, fully observable, and stationary environments, often neglecting to incorporate real-world complexities explicitly. In this paper, we introduce \texttt{Gym4ReaL}, a comprehensive suite of realistic environments designed to support the development and evaluation of RL algorithms that can operate in real-world scenarios. The suite includes a diverse set of tasks that expose algorithms to a variety of practical challenges. Our experimental results show that, in these settings, standard RL algorithms confirm their competitiveness against rule-based benchmarks, motivating the development of new methods to fully exploit the potential of RL to tackle the complexities of real-world tasks.