Systematic Comparison of Path Planning Algorithms using PathBench

TL;DR

This paper introduces PathBench, a comprehensive platform for developing, visualizing, and benchmarking various classical and learning-based path planning algorithms across different environments and hardware, facilitating fair comparison and analysis.

Contribution

The paper presents PathBench, an open-source, unified platform that supports development, visualization, and benchmarking of diverse path planning algorithms in 2D and 3D environments.

Findings

PathBench enables comparison of algorithms across multiple hardware and map types.

Metrics like path length, success rate, and computational time are effectively used for evaluation.

PathBench supports real-world robot testing via ROS integration.

Abstract

Path planning is an essential component of mobile robotics. Classical path planning algorithms, such as wavefront and rapidly-exploring random tree (RRT) are used heavily in autonomous robots. With the recent advances in machine learning, development of learning-based path planning algorithms has been experiencing rapid growth. An unified path planning interface that facilitates the development and benchmarking of existing and new algorithms is needed. This paper presents PathBench, a platform for developing, visualizing, training, testing, and benchmarking of existing and future, classical and learning-based path planning algorithms in 2D and 3D grid world environments. Many existing path planning algorithms are supported; e.g. A*, Dijkstra, waypoint planning networks, value iteration networks, gated path planning networks; and integrating new algorithms is easy and clearly specified. The benchmarking ability of PathBench is explored in this paper by comparing algorithms across five different hardware systems and three different map types, including built-in PathBench maps, video game maps, and maps from real world databases. Metrics, such as path length, success rate, and computational time, were used to evaluate algorithms. Algorithmic analysis was also performed on a real world robot to demonstrate PathBench's support for Robot Operating System (ROS). PathBench is open source.