Motion Planning as Online Learning: A Multi-Armed Bandit Approach to Kinodynamic Sampling-Based Planning

TL;DR

This paper introduces a novel kinodynamic motion planning method that uses a multi-armed bandit framework to adaptively bias sampling, resulting in faster, higher-quality solutions for complex robotic tasks with dynamics constraints.

Contribution

It formulates the sampling bias problem as a non-stationary multi-armed bandit, enabling adaptive and efficient exploration in kinodynamic planning.

Findings

Faster convergence to high-quality solutions

Higher success rate in complex manipulation tasks

Effective in scenarios with dynamics uncertainty

Abstract

Kinodynamic motion planners allow robots to perform complex manipulation tasks under dynamics constraints or with black-box models. However, they struggle to find high-quality solutions, especially when a steering function is unavailable. This paper presents a novel approach that adaptively biases the sampling distribution to improve the planner's performance. The key contribution is to formulate the sampling bias problem as a non-stationary multi-armed bandit problem, where the arms of the bandit correspond to sets of possible transitions. High-reward regions are identified by clustering transitions from sequential runs of kinodynamic RRT and a bandit algorithm decides what region to sample at each timestep. The paper demonstrates the approach on several simulated examples as well as a 7-degree-of-freedom manipulation task with dynamics uncertainty, suggesting that the approach finds better solutions faster and leads to a higher success rate in execution.