Growing Trees with an Agent: Accelerating RRTs with Learned, Multi-Step Episodic Exploration

TL;DR

This paper introduces Episodic RRT, a hybrid motion planning method that uses a learned, multi-step exploration agent to significantly improve efficiency and success rates in high-dimensional and cluttered spaces.

Contribution

It presents a novel framework replacing random sampling with a learned exploration policy, enhancing directed growth and reducing collision checks in RRT-based planners.

Findings

Achieves 98% success rate in 6D robotic arm scenarios

Up to 107x faster than classical RRT in complex environments

Reduces collision checks by over 99.6%

Abstract

Classical sampling-based motion planners like the RRTs suffer from inefficiencies, particularly in cluttered or high-dimensional spaces, due to their reliance on undirected, random sampling. This paper introduces the Episodic RRT, a novel hybrid planning framework that replaces the primitive of a random point with a learned, multi-step "exploratory episode" generated by a Deep Reinforcement Learning agent. By making the DRL agent the engine of exploration, ERRT transforms the search process from a diffuse, volumetric expansion into a directed, branch-like growth. This paradigm shift yields key advantages: it counters the curse of dimensionality with focused exploration, minimizes expensive collision checks by proactively proposing locally valid paths, and improves connectivity by generating inherently connected path segments. We demonstrate through extensive empirical evaluation across 2D, 3D, and 6D environments that ERRT and its variants consistently and significantly outperform their classical counterparts without any GPU acceleration. In a challenging 6D robotic arm scenario, ERRT achieves a 98% success rate compared to 19% for RRT, is up to 107x faster, reduces collision checks by over 99.6%, and finds initial paths that are nearly 50% shorter. Furthermore, its asymptotically optimal variant, ERRT*, demonstrates vastly superior anytime performance, refining solutions to near-optimality up to 29x faster than standard RRT* in 3D environments. Code: https://xinyuwuu.github.io/Episodic_RRT/.