Arena-Bench: A Benchmarking Suite for Obstacle Avoidance Approaches in Highly Dynamic Environments

TL;DR

Arena-bench is a comprehensive benchmarking suite designed for training, testing, and evaluating obstacle avoidance methods for autonomous robots in highly dynamic 3D environments, facilitating comparison and deployment of diverse approaches.

Contribution

The paper introduces Arena-bench, a new benchmark suite that supports dynamic environment creation, integration with ROS, and real robot deployment for obstacle avoidance research.

Findings

DRL agent trained on Arena-bench outperforms some existing methods.

Benchmark suite enables reproducible evaluation across different platforms.

Approaches successfully deployed on real robots demonstrating practical applicability.

Abstract

The ability to autonomously navigate safely, especially within dynamic environments, is paramount for mobile robotics. In recent years, DRL approaches have shown superior performance in dynamic obstacle avoidance. However, these learning-based approaches are often developed in specially designed simulation environments and are hard to test against conventional planning approaches. Furthermore, the integration and deployment of these approaches into real robotic platforms are not yet completely solved. In this paper, we present Arena-bench, a benchmark suite to train, test, and evaluate navigation planners on different robotic platforms within 3D environments. It provides tools to design and generate highly dynamic evaluation worlds, scenarios, and tasks for autonomous navigation and is fully integrated into the robot operating system. To demonstrate the functionalities of our suite, we trained a DRL agent on our platform and compared it against a variety of existing different model-based and learning-based navigation approaches on a variety of relevant metrics. Finally, we deployed the approaches towards real robots and demonstrated the reproducibility of the results. The code is publicly available at github.com/ignc-research/arena-bench.