RobocupGym: A challenging continuous control benchmark in Robocup

TL;DR

RobocupGym introduces a new continuous control benchmark in a robotic football simulation, facilitating reinforcement learning research with high-dimensional tasks involving simulated Nao robots, and is openly available for the community.

Contribution

We developed and open-sourced a Robocup-based RL environment with high-dimensional control tasks, integrating it with Stable Baselines 3 for easier application of RL in robotics football.

Findings

Enables high-dimensional continuous control in Robocup simulation

Provides a standardized environment for RL benchmarking in robotics

Open-source code promotes community adoption and further research

Abstract

Reinforcement learning (RL) has progressed substantially over the past decade, with much of this progress being driven by benchmarks. Many benchmarks are focused on video or board games, and a large number of robotics benchmarks lack diversity and real-world applicability. In this paper, we aim to simplify the process of applying reinforcement learning in the 3D simulation league of Robocup, a robotic football competition. To this end, we introduce a Robocup-based RL environment based on the open source rcssserver3d soccer server, simple pre-defined tasks, and integration with a popular RL library, Stable Baselines 3. Our environment enables the creation of high-dimensional continuous control tasks within a robotics football simulation. In each task, an RL agent controls a simulated Nao robot, and can interact with the ball or other agents. We open-source our environment and training code at https://github.com/Michael-Beukman/RobocupGym.