Comparing Popular Simulation Environments in the Scope of Robotics and Reinforcement Learning

TL;DR

This paper compares four popular robotics simulation environments for reinforcement learning, analyzing their performance across different hardware setups to guide efficient selection and deployment in industrial applications.

Contribution

It provides a comprehensive benchmark of simulation environments considering hardware variations, highlighting the importance of single-core performance and parallelization for RL training efficiency.

Findings

Single-core performance significantly impacts simulation speed.

Parallel simulations can improve throughput on multi-core systems.

Performance varies notably across different simulation environments.

Abstract

This letter compares the performance of four different, popular simulation environments for robotics and reinforcement learning (RL) through a series of benchmarks. The benchmarked scenarios are designed carefully with current industrial applications in mind. Given the need to run simulations as fast as possible to reduce the real-world training time of the RL agents, the comparison includes not only different simulation environments but also different hardware configurations, ranging from an entry-level notebook up to a dual CPU high performance server. We show that the chosen simulation environments benefit the most from single core performance. Yet, using a multi core system, multiple simulations could be run in parallel to increase the performance.