An Extensible Benchmarking Infrastructure for Motion Planning Algorithms

TL;DR

This paper introduces an extensible benchmarking infrastructure integrated with OMPL, enabling systematic evaluation and visualization of motion planning algorithms across diverse problem classes.

Contribution

It presents a comprehensive benchmarking framework, including software, data formats, and visualization tools, for analyzing motion planning algorithms' performance.

Findings

Benchmarking software integrated with OMPL

Extensible formats for storing results

Interactive visualization tool for data analysis

Abstract

Sampling-based planning algorithms are the most common probabilistically complete algorithms and are widely used on many robot platforms. Within this class of algorithms, many variants have been proposed over the last 20 years, yet there is still no characterization of which algorithms are well-suited for which classes of problems. This has motivated us to develop a benchmarking infrastructure for motion planning algorithms. It consists of three main components. First, we have created an extensive benchmarking software framework that is included with the Open Motion Planning Library (OMPL), a C++ library that contains implementations of many sampling-based algorithms. Second, we have defined extensible formats for storing benchmark results. The formats are fairly straightforward so that other planning libraries could easily produce compatible output. Finally, we have created an interactive, versatile visualization tool for compact presentation of collected benchmark data. The tool and underlying database facilitate the analysis of performance across benchmark problems and planners.